• Computer Science > Computation and Language [Submitted on 5 Feb 2026] Title:TRACE: Trajectory-Aware Comprehensive Evaluation for Deep Research Agents View PDF HTML (experimental)Abstract:The evaluation of Deep Research Agents is a critical challenge, as conventional outcome-based metrics fail to capture the nuances of their complex reasoning • Current evaluation faces two primary challenges: 1) a reliance on singular metrics like Pass@1, creating a “high-score illusion” that ignores the quality, efficiency, and soundness of the reasoning process; and 2) the failure of static benchmarks to quantify crucial attributes like robustness and latent capability • To address these gaps, we introduce TRACE (Trajectory-Aware Comprehensive Evaluation), a framework that holistically assesses the entire problem-solving trajectory • To counter the “high-score illusion”, we propose a Hierarchical Trajectory Utility Function that quantifies process efficiency and cognitive quality, including evidence grounding, alongside accuracy • To measure deeper attributes, TRACE introduces a Scaffolded Capability Assessment protocol, quantifying an agent’s latent ability by determining the minimum guidance needed

Article Summaries:

  • Computer Science > Computation and Language [Submitted on 5 Feb 2026] Title:TRACE: Trajectory-Aware Comprehensive Evaluation for Deep Research Agents View PDF HTML (experimental)Abstract:The evaluation of Deep Research Agents is a critical challenge, as conventional outcome-based metrics fail to capture the nuances of their complex reasoning. Current evaluation faces two primary challenges: 1) a reliance on singular metrics like Pass@1, creating a “high-score illusion” that ignores the quality, efficiency, and soundness of the reasoning process; and 2) the failure of static benchmarks to quant

Sources: