Sign in

Category: Other

  • The Format is the Filter: Scoring Format Fixes What an LLM Judge Can Express

    The Format is the Filter: Scoring Format Fixes What an LLM Judge Can Express

    arXiv:2610.17158v1 Announce Type: new Abstract: When an LLM judge filters instruction-tuning data, its scoring format (binary verdict, 1–5 rating, 0–100 score, or comparison) is rarely justified or ablated. Holding the candidate pool, judge prompt, decoding and selection budget fixed, we score 15,406 UltraFeedback candidates under five formats with three judges, curate, fine-tune 66 models, and evaluate them with four judges. A 0–100 scale is worth only 7–12 equally used levels, two to three times a 1–5 scale rather than twenty, and a binary verdict leaves the top score tied in 77–78\% of prompts, so most selections are made at random. These properties are set by the format almost independently of the judge. Kendall's $\tau_b$ hides them: it ranks binary best while $\tau_a$ and top-1 agreement rank it last, because it discounts the ties a selector must break at random. Both comparative formats inherit a slot bias whose sign differs across judges. Downstream, curated data beat random selection under every evaluation judge, robust to length and markdown adjustment, but no ordering among formats is detectable. We propose four quantities that judge-based curation pipelines should report.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.17158

  • Neos: Learning to Interleave Search and Visual Reasoning

    Neos: Learning to Interleave Search and Visual Reasoning

    Neos: Learning to Interleave Search and Visual Reasoning

    arXiv:2610.17043v1 Announce Type: new Abstract: Recent multimodal deep search agents have advanced tool use and visual content retrieval, yet they still struggle to convert intermediate visual evidence into actionable guidance for subsequent search, limiting their effectiveness in long-horizon settings. To address this gap, we present Neos, a multimodal deep search framework that learns both how to acquire evidence and how to use it to guide subsequent decisions. To make this coupling learnable, Neos constructs training tasks with intermediate visual dependencies while filtering out textual shortcuts, and introduces a lightweight, trainable Multimodal Evidence Interface (MEI) that transforms retrieved content into goal-conditioned observations. At its core, an agent-environment co-evolution procedure treats MEI as a co-adaptive component of this loop. Specifically, reference-guided credit assignment uses verified successful trajectories to localize critical policy errors while preserving credit for useful decisions in failed searches, whereas rollout feedback selects informative local evidence-processing interactions for MEI supervised fine-tuning (SFT). By alternating policy reinforcement learning (RL) with MEI SFT, Neos allows the policy's evolving evidence requests to shape MEI training and the adapted MEI to reshape subsequent observations, closing the loop between evidence acquisition and evidence-conditioned action. Across ten benchmarks, Neos-9B and Neos-35B demonstrate leading performance among specialized multimodal deep-search agents.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.17043

  • VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval

    VL-DocIR: A Benchmark for Vision-Based Long Document Information Retrieval

    arXiv:2610.17057v1 Announce Type: new Abstract: Vision-based document retrieval has improved rapidly, but current evaluation settings still emphasize short documents and single relevant pages. This leaves an important gap between leaderboard performance and realistic long-document retrieval, where the evidence needed to answer a query may span multiple pages or even multiple documents. We present VL-DocIR, a page-level benchmark for evidence-complete visual long-document retrieval. VL-DocIR is built from 29,641 HTML-born documents rendered into 388,548 page images from Wikipedia, arXiv, PubMed, and SEC proxy statements. It contains 271,760 queries across 23 domains and six evidence structures, covering single-page, within-document multi-page, and cross-document evidence configurations. Evidence annotations are grounded to rendered pages and HTML element identifiers, and queries are filtered with a cleaning pipeline targeting clarity, correctness, closedness, and the absence of explicit layout references. We evaluate leading single-vector and multi-vector visual retrievers together with diagnostic retrieval settings such as document-context scoring, query splitting, reranking, and round-robin ensembles. The strongest evaluated configuration reaches 69.3 nDCGAll@10 and 80.6 RecallAll@10 under structure-balanced Macro evaluation, but source- and evidence-structure breakdowns show that cross-document retrieval remains substantially harder than single-page retrieval, and that document-context scoring is especially helpful for within-document multi-page queries. VL-DocIR exposes these failure modes and provides a more demanding target for visual long-document retrieval. The new dataset and code will be made publicly available at https://yufanchen96.github.io/VL-DocIR.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.17057

  • PatientWM: Towards Generalizable Clinical World Models via Unified Patient States

    PatientWM: Towards Generalizable Clinical World Models via Unified Patient States

    PatientWM: Towards Generalizable Clinical World Models via Unified Patient States

    arXiv:2610.17059v1 Announce Type: new Abstract: Clinical decisions depend on anticipating how a patient's condition will evolve as treatment is adjusted and new evidence becomes available. Existing medical world models capture patient-state evolution, but often define the evolving state around a particular modality or task, limiting reuse as inputs or prediction targets change. These models also retain history through observation sequences or recurrent or latent summaries rather than exposing preceding patient states and clinical actions as an ordered transition context. This distinction matters because similar current measurements may follow different recent changes, treatment exposures, and observed responses. To tackle these limitations, we propose PatientWM, a clinical world-model framework built around a unified patient state and sequential latent history. Availability-aware encoders integrate observations available at each decision time into the patient state, while typed decoders produce task-specific clinical readouts without defining the state around any single target. PatientWM further conditions each next-state prediction on the current state and supplied action together with a bounded, ordered history of preceding latent patient states and clinical actions, preserving recent treatment and response context. Experiments across MIMIC-III, MIMIC-IV, and three external clinical datasets show superior performance in clinical prediction, multi-step forecasting, and action-planning evaluation. Transfer experiments further demonstrate the generalizability of PatientWM's patient state. Together, these results establish its ability to forecast clinical evolution and support planning.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.17059

  • Glasseek: A Transparent Recipe for Training and Trustworthy Evaluation of DeepSearch Agents

    Glasseek: A Transparent Recipe for Training and Trustworthy Evaluation of DeepSearch Agents

    Glasseek: A Transparent Recipe for Training and Trustworthy Evaluation of DeepSearch Agents

    arXiv:2610.16998v1 Announce Type: new Abstract: Deep search enables language agents to solve complex information-seeking tasks through iterative web exploration. However, developing robust open-source agents remains challenging for three reasons. First, opaque trajectory-construction pipelines make training data difficult to audit and reproduce. Second, context compression can create a substantial mismatch between training and inference. Third, agents may retrieve benchmark content from the open web during evaluation, contaminating the results and overstating their performance. To overcome these bottlenecks, we introduce Glasseek, a fully transparent deep search framework spanning the complete development lifecycle, releasing all data, model weights, training recipes, and code. Specifically, for data construction, we establish a verifiable trajectory curation pipeline that systematically generates high-quality exploration paths. For training, we propose Compaction-Managed GSPO (CM-GSPO), which uses the same policy for search and summarization while optimizing only execution sub-trajectories, enabling end-to-end search-policy training across context-compaction boundaries. For evaluation, we conduct the first systematic audit of test-set leakage in open-web agents and release a modular anti-leakage plugin that seamlessly integrates into existing evaluation harnesses. Under this rigorous evaluation protocol, our 9B agent outperforms frontier proprietary agents across almost all deep research benchmarks spanning diverse task types, and achieves the best overall performance among recent open-weight agents, setting a new state of the art at the 9B scale.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16998

  • Understanding Personalization Gains in Whisper

    Understanding Personalization Gains in Whisper

    Understanding Personalization Gains in Whisper

    arXiv:2610.17019v1 Announce Type: new Abstract: Whisper-style ASR models provide strong zero-shot transcription, but their residual errors are not evenly distributed across datasets, recording conditions, and metadata-based groups. We study when lightweight low-rank adaptation (LoRA) improves Whisper and whether observed gains require group-specific adapters. Across Common Voice, LibriSpeech, and VoxPopuli, we compare zero-shot Whisper, pooled LoRA, and group-specific LoRA under matched within-group splits. We find that adaptation is not uniformly beneficial: gains are concentrated in difficult Common Voice groups, are consistent on VoxPopuli, and largely disappear on low-error LibriSpeech. Crucially, pooled LoRA often matches, and sometimes clearly outperforms, group-specific LoRA. Gains over zero-shot should therefore be tested against matched pooled adaptation before being interpreted as evidence of personalization.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.17019

  • Benchmarking Quantum Kernel SVMs for Credit Card Fraud Detection: Performance Realities Under NISQ Constraints

    Benchmarking Quantum Kernel SVMs for Credit Card Fraud Detection: Performance Realities Under NISQ Constraints

    Benchmarking Quantum Kernel SVMs for Credit Card Fraud Detection: Performance Realities Under NISQ Constraints

    arXiv:2610.17027v1 Announce Type: new Abstract: Detecting credit card fraud is notoriously difficult because fraudulent charges make up less than 0.2% of total transactions. Quantum Machine Learning (QML), particularly Quantum Kernel Support Vector Machines (QSVM), promises richer feature representations by embedding classical data into high-dimensional Hilbert spaces. However, whether this theoretical advantage holds up when managing real-world financial data bottlenecks remains unclear. We designed a controlled benchmark comparing three quantum encodings, Angle, ZZFeatureMap, and IQP against classical models while keeping sample sizes, feature counts, and scaling strictly matched. To handle class skew without corrupting quantum state overlaps, we embedded synthetic minority oversampling (SMOTE) directly into the quantum Gram matrix workflow, raising PR-AUC scores by roughly +0.04 across feature maps. When trained on the full European credit card dataset with 30 features, classical baselines reached a top PR-AUC of 0.983. However, restricting classical models to match QSVM hardware constraints (500 samples, 4 PCA features) dropped their PR-AUC to 0.750. This puts our best quantum result 0.730 PR-AUC using IQP encoding on 150 samples into realistic context. Testing on physical quantum hardware reduced performance further to 0.62 due to CNOT gate errors and thermal noise. Overall, our findings pinpoint two main operational bottlenecks for near-term quantum kernels: the scaling of Gram matrix evaluation and heavy information loss from squeezing features onto small qubit registers.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.17027

  • Transparent Information Flow Modeling for Role-Aware AI Act Assessment

    Transparent Information Flow Modeling for Role-Aware AI Act Assessment

    Transparent Information Flow Modeling for Role-Aware AI Act Assessment

    arXiv:2610.16916v1 Announce Type: new Abstract: The European Union's Artificial Intelligence Act introduces a structured framework for governing high-risk AI systems, emphasizing transparency, risk management, and role-specific accountability. However, current assessment practices often rely on monolithic system views or disconnected checklists, limiting their ability to capture the socio-technical complexity of real-world deployments. We explore the use of the Information Flow Methodology (IFM), a modeling-based approach to AI Act compliance assessment. IFM represents decision systems — whether algorithmic, human, or hybrid — as networks of information sites and transformation channels, enabling structured analysis of how information, decisions, and responsibilities flow across organizational and technical boundaries. We apply IFM in a real-world case involving a recruitment support tool used by a public-sector organization, evaluating both the Provider's system and the Deployer's recruitment process. Our results demonstrate that IFM can serve as a scaffold to bridge both component-based and holistic perspectives, and how technical systems are situated within socio-technical contexts. In doing so, IFM supports transparency, identifies risk pathways, and enables alignment analysis between intended and actual use. We argue that such modeling provides a foundational tool for effective, context-sensitive AI governance and accountability — offering capabilities beyond conventional checklist-based assessments.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16916

  • WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search

    WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search

    WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search

    arXiv:2610.16927v1 Announce Type: new Abstract: Short-video search requires compact query rewriters that infer personalized intent from brief, ambiguous queries under strict latency constraints. The challenge is to decide when personalization is warranted and generate a reformulation that preserves the user's immediate information need. A rewriter optimized for fluency or engagement may still introduce unsupported details and redirect the search toward a different intent. Existing methods often underutilize users' actual reformulations and behavioral traces, relying instead on synthetic reasoning and indirect behavioral rewards that may produce more engaging candidates but cause intent drift. To build an effective rewriter that remains faithful to user intent, we propose WeFair, Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in WeChat short-video search. WeFair assigns real user behavioral signals to a progression from demonstration to preference to verification. We first mine actual human reformulations from search sessions, then apply multidimensional LLM screening and contextual verification to filter unsupported transitions without extensive manual annotation. During training, we begin with a supervised curriculum that progressively builds rewriting capabilities from large-scale human personalized query demonstrations, and then use direct preference optimization to align with users' subsequent consumption. In the final on-policy optimization stage, we construct a strict verification reward based on actual human reformulations to further boost performance with group relative policy optimization. Further analysis shows that auxiliary rewards can activate more rollout groups in GRPO but reduce intent fidelity. Online A/B testing in WeChat short-video search shows a 1.576% relative increase in effective clicked-query rate (VV>10s) and a 1.731% relative reduction in query reformulation rate.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16927

  • MADDIE: Multi-Agent Drug Discovery Engine

    MADDIE: Multi-Agent Drug Discovery Engine

    MADDIE: Multi-Agent Drug Discovery Engine

    arXiv:2610.16975v1 Announce Type: new Abstract: Drug discovery remains a resource-intensive and time-consuming process, involving billions of dollars and a decade of development to advance a drug to market. While many in silico tools have accelerated this process, operating them still demands substantial time, effort, and expertise from medicinal chemists. Autonomous systems built on Large Language Models (LLMs) have emerged as a promising solution, yet current designs limit their performance. Existing single-agent systems scale poorly to long-horizon, multi-step, and realistic tasks, whereas multi-agent systems typically execute predefined workflows that cannot capture the iterative, non-linear nature of a real drug discovery campaign. Here, we present MADDIE, a multi-agent drug discovery engine. It comprises a central agent, Orchestrator, and four specialized worker agents: HitIdentifier, LeadOptimizer, Synthesizer, and Evaluator, each representing a critical step in a drug discovery campaign. Under Orchestrator's orchestration and coordination, these worker agents are composed into flexible workflows, enabling MADDIE to adapt its strategy and navigate the drug discovery pipeline end-to-end. Across 50 protein targets, MADDIE produces molecules satisfying key property requirements and their synthesis routes, outperforming existing agentic systems, generative models, and general-purpose LLMs by a wide margin. Our analysis reveals how these agents collaborate and that MADDIE can employ different strategies across targets to achieve the campaign's goal, while still being efficient in time and tokens. Together, they demonstrate MADDIE as a low-cost, effective, and efficient autonomous engine for accelerating drug discovery. Code and dataset is available via https://github.com/ninglab/MADDIE.

    Fallback brief based on the available RSS feed; no unsupported details have been added.

    Source: arXiv
    Original article: https://arxiv.org/abs/2610.16975