Sign in

Category: Other

  • Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization

    Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization

    Same Operation, Different Permission: Evaluating Whether Guardrails Honor Authorization

    arXiv:2610.16838v1 Announce Type: new Abstract: As language-model agents write code, move money and change records on behalf of users, deciding whether an action is authorized has become a central safety problem. Guardrails are often placed in front of these actions, yet few evaluations ask whether their decisions follow authorization: blocking unauthorized actions while still allowing the same actions under valid permission. We introduce \bench{}, a benchmark that contrasts authorization evidence for the same operation in its language track. It combines expert-annotated natural-language scenarios with executed and verified operations, plus externally sourced native-policy requests. \bench{} checks each decision for correctness, usability and resulting effect, and tracks each request across guardrail changes. Our experiments show that many blocked operations stem from failed reviews, not correct denials. Repairing the output format restores usable decisions but changes which authorized requests are honored, and adding a second reviewer removes some errors while rejecting valid authorization. Guardrails should therefore be evaluated on both the unauthorized actions they stop and the authorized actions they let through.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16838

  • Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision

    Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision

    Trace-to-Architecture: Capability Bottleneck Diagnosis for Harness Revision

    arXiv:2610.16725v1 Announce Type: new Abstract: Improving an LLM agent requires knowing what to change when it fails. Similar failures can require different fixes, and resolving one problem can expose another. We introduce Trace-to-Architecture (T2A), a framework that uses execution trajectories to determine what to change in an agent's harness and when those changes apply. A development agent diagnoses failures using execution evidence from successful and failed tasks, revises the corresponding harness components, and then checks whether target tasks improve, selected successful tasks remain successful, and the edits are actually triggered. New execution evidence guides subsequent diagnosis and revision. Across ALFWorld, ScienceWorld, and AppWorld, T2A achieves the highest aggregate score for all three evaluated task agents, exceeding their respective strongest baselines by 6.2–30.3 points. We also integrate T2A's diagnosis and validation procedures into Meta-Harness, improving the resulting harnesses' mean ScienceWorld score by 21.6 points from the same starting harness.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16725

  • Queue-Aware Token Communications: Bridging AI Representation and Wireless Service

    Queue-Aware Token Communications: Bridging AI Representation and Wireless Service

    Queue-Aware Token Communications: Bridging AI Representation and Wireless Service

    arXiv:2610.16730v1 Announce Type: new Abstract: The rapid advancement of foundation models and edge intelligence is transforming wireless networks from data-oriented communication systems into artificial intelligence (AI)-native service platforms. Token communications (TokenCom) has emerged as a promising paradigm that directly transmits task-oriented AI representations. However, existing studies typically assume that generated tokens can be transmitted immediately, overlooking the queueing dynamics arising from the mismatch between token workload generation and time-varying wireless service capability. To address this limitation, we introduce Queue-Aware TokenCom, which explicitly incorporates token queues into the AI service pipeline. We present an end-to-end architecture integrating AI representation generation, token workload management, wireless transmission, and downstream task inference. We then discuss two key design directions: budget-aware token representation optimization and queue-aware token service optimization, with the token budget serving as a critical interface between AI performance and communication efficiency. Moreover, we investigate fixed and adaptive token budget strategies to characterize the fundamental tradeoff between AI performance and communication efficiency in dynamic wireless environments. Finally, we outline research directions toward closed-loop, task-aware, and scalable TokenCom systems.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16730

  • Separating Prompt Harmfulness from Interaction Safety via Partial Generation

    Separating Prompt Harmfulness from Interaction Safety via Partial Generation

    Separating Prompt Harmfulness from Interaction Safety via Partial Generation

    arXiv:2610.16761v1 Announce Type: new Abstract: LLM-based information systems need safety guardrails. Most guardrails classify the prompt alone. If the prompt looks harmful, the system rejects it before any answer is generated. This is costly. Many rejected prompts would have received a safe response. Users lose help they should have received from the information system. This cost grows as conversational and generative search becomes common. Prompt-level labels treat harm as a property of the query. But harm depends on what the model actually writes. Existing input-output guards do look at responses. But they judge a finished response, so the decision comes after full generation. However, we observe that safety of model interaction with the input prompt can be inferred early with only partial output. So the system can start generating, watch the partial response, and stop only if it turns harmful. We test this idea in three steps. First, we build a dataset of 41,944 prompt-response pairs with seven open weights LLMs from three families: Llama, Gemma, and Qwen. Second, we show that prompt harmfulness and interaction safety differ. Prompt labels and response labels agree on only 50.46\% of items (Cohen's $\kappa$ = 0.06). Of the prompts that make any model respond harmfully, only 3.5\% do so for all seven LLMs. Third, we show that partial responses carry most of the signal. Fine-tuned classifiers predict response-level harm well (best AUPRC 0.885, F1 0.807). About one third of the response tokens recover almost full-response AUPRC.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16761

  • LATCH: LLM-Assisted Timing Closure Harness for Post-Route FPGA Optimisation

    LATCH: LLM-Assisted Timing Closure Harness for Post-Route FPGA Optimisation

    LATCH: LLM-Assisted Timing Closure Harness for Post-Route FPGA Optimisation

    arXiv:2610.16625v1 Announce Type: new Abstract: Post-route FPGA timing closure remains expert-dependent and computationally expensive. We present LATCH (LLM-Assisted Timing Closure Harness), a two-layer framework that separates large language model (LLM) orchestration from deterministic FPGA implementation and evaluation. Across sixteen designs and seven LLMs, a single-iteration study compares three agent control granularities – expert-written recipes, direct EDA operations and LLM-constructed procedures – to ask how much control an agent needs per attempt. Functional-pass rates are 98.2%, 71.4% and 17.9%, respectively; yet direct operations give slightly larger 3.4% median Fmax gains when passing, versus 3.2% for recipes. This exposes a trade-off between execution reliability and timing improvement. Informed by this trade-off, a multi-iteration study fixes the action tier at expert recipes to ask how repeated attempts are best organised, comparing a fitted deterministic heuristic with sequential LLM reasoning, branching exploration and knowledge base-guided exploration within 12 checkpoints per run. Branching raises mean maximum-frequency improvement from 10.5% (sequential) to 16.0%, and the knowledge base to 20.3%, improving 15 of 16 designs. Characterisation across 2,421 recipe attempts profiled their gain and validation rates, showing hard-block re-placement gives the largest cumulative mean gain at 41.4MHz.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16625

  • Does a Hosted Decision Model Respect Irrelevant Alternatives? A Pre-Registered Test of Option-Set Dependence in Jev

    Does a Hosted Decision Model Respect Irrelevant Alternatives? A Pre-Registered Test of Option-Set Dependence in Jev

    Does a Hosted Decision Model Respect Irrelevant Alternatives? A Pre-Registered Test of Option-Set Dependence in Jev

    arXiv:2610.16655v1 Announce Type: new Abstract: Decision APIs that return a probability for each option are routinely used as if those probabilities were properties of the options themselves. That reading is valid only if the model satisfies independence of irrelevant alternatives (IIA): adding an option that is neither the right answer nor its strongest rival should leave the odds between those two unchanged. We test this in a pre-registered study of jev-1.13.0, a hosted decision model, using 299,463 calls on intent classification (BANKING77, CLINC150) and topic classification (AG News). Every added option moves the log-odds between the gold label and the strongest wrong label: the root-mean-square systematic shift is 0.20-0.52 nats, 1.2 to 3 times the run-to-run variation, even when the added option takes at most 0.1% of the probability mass. The shifts are directional. An option similar to the wrong answer draws probability from it (+0.10 nats toward the gold label). A paraphrase of the wrong answer produces an intermediate pattern: the gold label's probability is almost conserved, but the paraphrase receives only about a third of the wrong answer's share rather than the half that strict substitution implies. The distortion grows steadily with the logarithm of the number of options, reaching about 1 nat at 255 options, and off-domain padding pushes the odds against the gold label. In contrast, questions packed into one call do not interact (bias < 0.001). Scoring each option with an independent yes/no question removes the option-set dependence but picks the right answer 8.7 points less often. Jev sits between two local baselines, an option-independent scorer and a prompt readout. Its probabilities are best read as relative to the option list they were computed over.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16655

  • Structural Baselines Matter for Agent Failure Detection and Attribution

    Structural Baselines Matter for Agent Failure Detection and Attribution

    arXiv:2610.16661v1 Announce Type: new Abstract: An LLM agent using tools can fail silently. It stops before the task is done, raises no error, and answers fluently. Two kinds of monitor address this, one deciding whether a run failed and the other naming the step where failure became unavoidable. Much of the reported performance on both tasks comes from regularities that need no model. Failing runs are longer, and the step labelled decisive is usually the last. A rule that always names the last step is correct $67\%$ to $96\%$ of the time here, without reading the run at all. That ceiling reflects how the labels were written, not how agents break. On failures whose decisive step replay can confirm, it falls to near chance. Reporting gain over that rule therefore changes what a score means. Under that correction, methods tested here gain nothing on give-up failures, while several published systems keep a clear advantage on other splits. For detection, the agent's internal states beat transcript length and text similarity on the agent benchmark, and hold up where the text baseline does not. Precision stays low when failures are rare, so such a monitor should rank runs for review, not replace it.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16661

  • Double-Rollout Reinforcement Learning with Sampling-based Reward Estimation

    Double-Rollout Reinforcement Learning with Sampling-based Reward Estimation

    arXiv:2610.16593v1 Announce Type: new Abstract: Reinforcement learning with verifiable rewards (RLVR) implemented with Group Relative Policy Optimization (GRPO) has driven substantial progress in training reasoning language models. However, verifiable outcome rewards provide an imperfect measure of reasoning quality: a correct final answer may not be supported by the reasoning trace, whereas an incorrect rollout may contain a nearly correct reasoning trajectory. We present empirical evidence for this mismatch and hypothesize that the resulting reward noise contributes to instability in reinforcement learning for both reasoning accuracy and token efficiency. To address this credit assignment challenge, this paper proposes double-rollout reinforcement learning. For each sampled rollout, we truncate its reasoning trace, sample multiple continuations from the truncated prefix, and replace the original reward with the mean continuation reward, yielding a more robust estimate of reasoning quality. This double-rollout algorithm incurs only a moderate increase in compute cost and yet reaches substantially higher accuracies across math and coding tasks and across Qwen and Nemotron models than baseline algorithms with fewer RL steps. For example, using only DAPO data it trains Nemotron-3-Nano-30B-A3B to reach 95.4% on AIME 2026, versus 91.5% by GRPO and substantially above state of the art for this model size, within 30 RL steps. When combined with length-based rewards, the same algorithm further improves the accuracy-efficiency Pareto frontier over existing efficiency optimizations. These findings suggest that the double-rollout reward estimation is an effective and cost-efficient way to improve both the accuracy and token efficiency of reasoning RL.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16593

  • When 1+1 Backfires: Rethinking Relevance-Diversity Balancing for Sample-Aware Visual Token Reduction

    When 1+1 Backfires: Rethinking Relevance-Diversity Balancing for Sample-Aware Visual Token Reduction

    arXiv:2610.16615v1 Announce Type: new Abstract: Visual token compression has emerged as a critical technique for reducing the inference cost of Vision-Language Models. Existing methods select tokens based on text relevance, visual diversity, or a fixed combination of both—treating the relevance-diversity balance as a universal constant independent of the input. We conduct a systematic analysis to challenge this assumption. At the dataset level, grid-searching the optimal relevance-diversity weight per benchmark reveals substantial cross-task variation. More critically, we further maximize per-sample answer likelihood to recover sample-level oracle weights, finding that their distributions not only differ markedly across benchmarks but also exhibit high within-benchmark variance. Based on these observations, we propose SAVER, a framework for Sample-Aware relevance-diversity balanced Visual tokEn Reduction. We first derive a Geometric Intent Prior (GIP) from the spatial concentration of text-relevant tokens to infer a sample-specific relevance-diversity weight, and then apply this weight to jointly select tokens via a Weighted Text Centroid (WTC) for relevance scoring and a Visual Marginal Coverage Gain (VMCG) for diversity. SAVER requires no additional training and achieves sample-aware balancing entirely at inference time. Extensive experiments demonstrate that SAVER consistently outperforms state-of-the-art baselines across diverse benchmarks, retaining 96.5% of full-model performance on LLaVA-1.5-7B with only 11.1% of the original visual tokens, and further generalizing to LLaVA-NeXT and Mini-Gemini.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16615

  • Weak Segmentation-Guided GAN for~Realistic Color Edition

    Weak Segmentation-Guided GAN for~Realistic Color Edition

    arXiv:2610.16620v1 Announce Type: new Abstract: Editing the color of images in a realistic way finds many applications such as changing the perception of an image, data augmentation or film post processing. The design of an automatic tool is a complex and long addressed challenge. In particular, two properties are difficult to meet altogether: generating realistic results without artifacts and the possibility to precisely choose the future color. Conventional methods using segmentation and histogram matching maximize the controllability but also introduce a lack of realism or are complex to automate. On the contrary, GANs that specialize in realism are difficult to control. To overcome these challenges, we propose a novel GAN architecture leveraging any differentiable segmentation model. We demonstrate the genericness of our framework that presents state of the art results on different use cases. It generates images that look realistic while offering a precise color control.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16620