Sign in

Author: DEROBS

  • Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback

    Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback

    Single-step Retrosynthesis via Reinforcement Learning from Forward Feedback

    arXiv:2610.15141v1 Announce Type: new Abstract: Single-step retrosynthesis is a one-to-many inverse problem, yet most models are trained and evaluated against a single recorded reactant set. Reference-based training gives no explicit positive signal for unrecorded alternatives, and exact-match evaluation rejects non-matching predictions regardless of chemical plausibility. We propose Retrosynthesis via Reinforcement Learning from Forward Feedback (RetroRLFF), which optimizes reactant generation using both reference agreement and forward consistency with the target product. Starting from a supervised decoder-only Transformer, RetroRLFF generates complete reactant candidates and uses a frozen forward reaction model to predict products. A sequence-level reward combines reference agreement and forward consistency while penalizing invalid outputs. Direct Advantage-Based Policy Optimization updates the policy without human preference annotations or a preference-trained reward model. On USPTO-50K, RetroRLFF outperforms all 14 baselines in candidate-level round-trip accuracy at k = 1, 3, 5, and 10. With approximately 50 million parameters, it achieves 90.1% top-1 round-trip accuracy, exceeding R-SMILES, the strongest competing baseline on this metric, by 8.5 percentage points. Compared with its supervised policy, round-trip accuracy increases from 83.1% to 90.1%, while exact-match accuracy is maintained at 57.9% versus 57.5%. In an anonymized expert assessment of 131 randomly selected test cases, organic synthesis experts prefer RetroRLFF over EditRetro in 67.9% of comparisons. These results support forward feedback as a training signal for improving forward consistency while retaining top-1 reference recovery. Expert assessment provides complementary evidence of synthetic feasibility on the selected cases, although neither forward consistency nor expert preference establishes experimental feasibility.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15141

  • Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses

    Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses

    Harmony Harness: Multi-Objective Self-Evolution of Agent Harnesses

    arXiv:2610.15150v1 Announce Type: new Abstract: Recent work on self-evolving LLM agents treats the agent harness as an optimization target that can be iteratively improved through execution feedback. Existing approaches, however, largely optimize a single objective and evolve the harness as a monolithic artifact, limiting their ability to handle competing goals and to separate transferable design from environment-specific adaptation. We introduce Harmony Harness, a framework for multi-objective, compositional harness evolution. Harmony Harness formulates harness evolution as a multi-objective optimization problem and uses decomposition-based evolutionary search to maintain solutions across different regions of the Pareto frontier, avoiding reliance on a hand-designed structural diversity metric for repository-scale harnesses. In parallel, we introduce factorized evolution operators that separately refine general-purpose harness mechanisms and benchmark-specific adaptations, together with a dynamic composition workflow that integrates them at inference time. Across SecRepoBench and BaxBench, Harmony Harness consistently outperforms strong harness-evolution baselines, with gains exceeding 10% on both benchmarks. Our results show that explicitly modeling both objective trade-offs and harness structure yields substantially stronger and more transferable agent evolution than existing single-objective, monolithic approaches.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15150

  • Self-Evolving Visual Agent via Margin Reward

    Self-Evolving Visual Agent via Margin Reward

    Self-Evolving Visual Agent via Margin Reward

    arXiv:2610.15084v1 Announce Type: new Abstract: Visual agents that interleave reasoning with tool invocation have posed a promising direction for multimodal tasks. However, learning effective tool invocation usually relies on curated question-answer pairs or expert demonstrations at scale, which are expensive to acquire. In this study, we propose Marvo, a simple-yet-effective self-evolution approach that enables learning tool invocation from unlabeled images. Marvo consists of two agents: (1) a challenger that generates question-answer pairs from unlabeled images, and (2) a solver that learns tool invocation by solving questions through reinforcement learning. To encourage effective tool invocation, a good training question should be difficult enough to warrant tool use, yet remain within the solver's capabilities. To this end, we introduce a margin reward to the challenger, defined as solver's performance difference with and without access to tools. With the margin reward, we create an adaptive curriculum that encourages the solver to learn effective tool invocation. Across benchmarks covering agentic reasoning, visual search, mathematical, and general real-world reasoning, Marvo improves average accuracy by up to 7.1\% — demonstrating margin reward as a preferred way to scale question complexity without human supervision.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15084

  • Structural Chain-of-Thought: Reasoning with Executable Computational Graphs

    Structural Chain-of-Thought: Reasoning with Executable Computational Graphs

    Structural Chain-of-Thought: Reasoning with Executable Computational Graphs

    arXiv:2610.15107v1 Announce Type: new Abstract: Chain-of-thought prompting enables reasoning in large language models but produces explanations that are difficult to verify, expensive to generate, and not always faithful. We introduce a Structural Chain-of-Thought framework where models generate executable computational graphs instead of free-form reasoning. The final answer is obtained by deterministically executing the graph. The model is trained using graph-based reinforcement learning that rewards structural validity, answer correctness, and node overlap with reference graphs. Experiments across five benchmarks and four model families show that our method consistently improves accuracy over supervised graph training and can reduce generation length by up to 5-6x compared with standard GRPO and chain-of-thought supervised training. Our faithfulness experiments reveal that models trained with our approach tend to rely on deriving their answers from the generated graphs rather than producing post-hoc rationalizations as usually in chain-of-thought reasoning.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15107

  • GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?

    GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?

    GesBench: When and Where Do Multimodal Language Models Perceive Social Meaning?

    arXiv:2610.15129v1 Announce Type: new Abstract: Human social perception is incremental: we infer intent before an utterance is complete, revise our interpretation as new cues arrive, and attend to specific facial, gestural, or prosodic signals that shape social meaning. Current multimodal LLM benchmarks largely evaluate final-answer understanding over complete episodes, rewarding post-hoc recognition of social meaning rather than the unfolding perception required for real-time interaction. Drawing on interactionally embedded Gestalt principles of multimodal human communication, we introduce GesBench: a human-grounded benchmark for cue-level social perception in multimodal LLMs. GesBench uses cumulative sub-utterance clips to measure when models form stable interpretations of speaker intention and social affordance, and human gaze-guided corruption to evaluate whether regions attended by humans are functionally important for model judgments. Built from more than 6000 utterance-level clips from scripted and unscripted conversational datasets, GesBench includes human annotations of unfolding judgments and gaze on a representative subset. Across open and closed multimodal models, we find a clear human-model gap: humans form predictive understandings early and revise more often, while models often hedge or repeat across different clips. GesBench shifts social evaluation from whether models eventually recognize meaning to when and where that meaning is formed. Our codes are publicly available at https://anonymous.4open.science/status/GesBench-share-212C.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15129

  • Belief Embedding Tree Search

    Belief Embedding Tree Search

    Belief Embedding Tree Search

    arXiv:2610.15046v1 Announce Type: new Abstract: Beliefs in a Partially Observable Markov Decision Process (POMDP) are posterior distributions over the hidden state given the agent's history of actions and observations. They summarize everything the history says about the state, so planning in a POMDP is planning in a Markov decision process (MDP) over beliefs. Belief MDPs have continuous state spaces even for finite POMDPs, and exact planning in them is intractable in general. Planners must therefore approximate beliefs and estimate values. Monte Carlo planners do both inside a search tree, with an approximate belief at every node and a value estimate at every leaf. Approximations usually consist of sets of sampled states, with leaf values coming from rollouts, hand-designed bounds, or learned networks. Policies and values computed from these approximations are limited by their quality, and prior methods with learned networks rely on hand-chosen summaries of the sampled states. A belief representation should be computable from the parent node's representation through the POMDP's dynamics, let the search sample from it, and be learned so that it provides the information that policies and value functions need. We present Belief Embedding Tree Search (BETS), a Monte Carlo planner that meets these requirements by representing each node's belief with a generative distribution embedding (GDE) while updating it through the POMDP dynamics. BETS trains its belief model alongside its policy and value networks through experience, with the realized hidden state of the POMDP as the belief model's training target. We evaluate BETS on six configurations of RockSample, LightDark, and PocMan against BetaZero and three planners without learned components. BETS is competitive or leading on all six, while every baseline falls far behind on at least one. Where beliefs are highest-dimensional, it outperforms them by a large margin, even without search.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15046

  • Trace Evolution Bench: Measuring Whether Agents Improve from Their Own Environment Traces

    Trace Evolution Bench: Measuring Whether Agents Improve from Their Own Environment Traces

    Trace Evolution Bench: Measuring Whether Agents Improve from Their Own Environment Traces

    arXiv:2610.15055v1 Announce Type: new Abstract: Self-evolving agents are usually scored by endpoint Gain: held-out success after an update minus success before. That difference can credit a frozen rescore as evolution. We introduce Trace Evolution Bench (TEB), a protocol that separates that rescore from a context or weight update, together with three frozen 80/20/40 packs. An update is accepted only when validation does not fall, held-out identifiers never enter the update, and a text update is not ranked with a weight update. The primary scoreboard is coding, graded by hidden unit tests in a closed workspace. On that pack (40 held-out tasks, one family), DeepSeek v4 Pro under no_evolve never changes its policy yet moves 7.5, 5.0, and 12.5 percentage points across three seeds. Of nine text-update cells inspired by GEPA, ACE, and ExpeL, seven have positive excess above that seed's own band, and one gepa_llm seed moves down by the width of its band (-12.5, regression 0.48). GLM and MiniMax do not reproduce that seven-of-nine pattern. No method stays above its own band on every seed of all three models: GLM has two paired cells above their own bands, and MiniMax expel_llm seed 44 is a third (regression 0.30). The methods were fixed on a validation screen before held-out was read. Office, graded by rule-bounded delivery, and manufacturing table QA are separate packs: office DeepSeek does not exceed its own bands (widest 17.5 points), and manufacturing has no paired positive excess beyond its seed band.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15055

  • Exposing AI Inference as a Telco Network API over the 5G User Plane

    Exposing AI Inference as a Telco Network API over the 5G User Plane

    Exposing AI Inference as a Telco Network API over the 5G User Plane

    arXiv:2610.15062v1 Announce Type: new Abstract: Supporting edge Artificial Intelligence (AI) inference as a network service is challenging because current mobile systems can transport tensors but do not expose or understand the inference context that gives those tensors operational meaning. Prior studies have addressed this gap through dynamic partitioning, collaborative inference, and split pipelines; however, these solutions remain application-bound, network-blind, or context-free. This work proposes Inference on Demand (IoD), an inference-on-demand abstraction that exposes split inference through a CAMARA-like Application Program Interface (API) and carries a split-aware tensor context in-band across the 5G user plane path. We evaluate the IoD on a programmable user plane function (UPF)-based testbed using a partitionable visual AI workload, showing that adaptive split selection can operate as a network-facing service while avoiding the rigid cost profile of static configurations. Including our findings, split inference becomes more than offloading when the service intent, runtime partitioning, and tensor transport are made visible to the mobile data plane. This opens a path toward network-native AI services, in which programmable telco infrastructure coordinates computation, context, and connectivity for user-facing applications.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15062

  • Knowledge transfer between simple discrete worlds

    Knowledge transfer between simple discrete worlds

    Knowledge transfer between simple discrete worlds

    arXiv:2610.15081v1 Announce Type: new Abstract: Adapting to a new world and reasoning correctly about which outcomes are possible under rules one has not met before is a basic capacity underlying general intelligence. We study it in small discrete worlds: sets of grid configurations consistent with a shared set of rules, concretely Sudoku-like puzzles built from a common base and a handful of extra rules. We train a single depth-recurrent model on many combinations of rules and test it at four levels of novelty: a new puzzle, a new combination of known rules, a pairing of rules never seen together, and a rule instance not seen at all during training. The model receives the active rules either as a symbolic description or as two solved examples, and a model that gets no rule information is used as a baseline. At recombining rules it was trained on, the description matches or exceeds examples ($99\%$ against $95\%$ on new combinations). On rule instances whose components never co-occurred in training, the description falls below the no-information baseline while examples stay above it. The framework allows us to test many such generalization regimes in more systematic way than benchmarks such as ARC-AGI and provides new ground for evaluating depth-recurrent reasoning models, which are typically tested on saturated Sudoku benchmarks.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.15081

  • Certify the Decision, Not Every Score: Exact Top-k Selection from Few Evaluations

    Certify the Decision, Not Every Score: Exact Top-k Selection from Few Evaluations

    Certify the Decision, Not Every Score: Exact Top-k Selection from Few Evaluations

    arXiv:2610.14996v1 Announce Type: new Abstract: Choosing a few robust algorithms or schedules can require evaluating every candidate under many adverse scenarios, even though an agent only needs to identify the winners. A predictor may suggest those winners, but cannot rule out a surprise in an unevaluated scenario. We introduce Certified Adaptive Selection (CAS), which returns the exact top-$k$ set as soon as remaining uncertainty cannot change it. CAS keeps guaranteed lower and upper bounds on Expected Shortfall (ES), the average loss in the worst scenarios. Its stopping test compares the worst admissible selected candidate with the best admissible excluded one; for information-disjoint Cartesian intervals this one-cut test is necessary and sufficient. Prediction has a separate role: selecting the next outcome to evaluate, never authorizing the decision. For empirical ES with exact unit-cost queries, we compute in polynomial time the fewest evaluations that could have proved the realized decision in hindsight. Our prediction-guided agent CAS-PG is exact for every predictor, including adversarial ones. On two public algorithm-selection matrices, CAS-PG certifies the exact top three after 630 of 3150 and 981 of 2200 replayed queries, or $1.14$ and $1.18$ times that hindsight minimum. Without historical outcomes, its transcript-only variant requires $1.28$ and $1.37$ times the minimum, outperforming the tested non-predictive certified policies. Separately, a prediction-free CAS policy avoids $93.0\%$ of exact recourse solves in synthetic energy scheduling. One certificate thus supports several acquisition policies and separately proved risk/cost extensions: prediction changes the work, not the justified choice.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.14996