On November 13, 2026, researchers Disha Sheshanarayana and Sai Linisha Reddy Goli from Manipal University Jaipur released an empirical evaluation preprint examining whether large language model (LLM) verifiers maintain reliable pass-fail judgments when agent execution trajectories are transformed across composed tools, multi-agent delegation boundaries, or compacted context histories. Their study introduces Verification Transport, a reliability benchmark demonstrating that verifiers exhibiting strong native-view accuracy fail up to 31.0% of the time when exposed to standard architectural abstractions, with critical implications for AI safety auditing, benchmark design, and automated research oversight.
What the study discovered
In deployed artificial intelligence systems, autonomous agents rarely execute tasks within a flat, single-tier loop. Developers routinely encapsulate multi-step shell commands into composite macro tools, delegate subtasks to specialized worker agents, or compact conversation histories to manage token budgets. While these abstractions streamline execution, they alter what operational evidence is directly visible to the downstream verifier or process reward model tasked with judging whether the agent executed correctly.
The research paper, titled Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation (available on arXiv under arXiv:2611.18471), examines whether a verifier judgment that is correct under a complete native execution trace remains correct when that exact execution is viewed through an abstraction boundary. To isolate representation sensitivity from execution quality, the authors developed a matched evaluation framework: the underlying task, realized actions, final environment state, ground-truth label, evaluation criterion, and verifier model are held frozen, while only the observability condition presented to the verifier changes.
Across four established agent benchmarks and seven frontier verifier backbones, the authors uncovered extensive transport failure. Conditioning on judgments that were correct under native uncompressed observability, the denominator-weighted Verification Transport Failure Rate (VTFR) reached 31.0% under operation-level macro abstraction and 30.9% under agent delegation boundaries. Temporal history compaction, which retained recent tool calls and execution results, induced a 20.3% failure rate. Across benchmarks, macro abstraction produced aggregate failure rates between 25.5% and 32.2%, while delegation produced failures between 26.1% and 32.9%.
Experimental setup and benchmark evaluation
The authors evaluated Verification Transport across four heterogeneous agent evaluation benchmarks representing diverse operational challenges:
- AgentProcessBench (APB): 250 trajectories from the diagnostic benchmark developed by Fan et al. (2026) (arXiv:2603.14465), evaluating step-level process correctness rather than coarse terminal success.
- ATBench: 758 tool-use trajectories from Li et al. (2026) (arXiv:2604.02022), assessing real-world tool execution and parameter selection.
- PlanRewardBench (PRB): 441 pairwise trajectory comparisons evaluating reward modeling and planning judgment from Wang et al. (2026).
- AgentDojo: 754 security trajectories evaluated under dynamic prompt-injection attacks by Debenedetti et al. (2024), published on OpenReview.
The experiment evaluated seven verifier backbones across major model families: Gemini-3.7-Flash, Gemini-2.5-Flash, GPT-OSS-120B, GPT-OSS-20B, Gemma-4-31B, Gemma-4-26B-A4B, and Llama-4-Maverick. All models were queried at temperature zero with enforced structured outputs to prevent sampling variance from confounding judgment comparisons.
The table below summarizes native operational balanced accuracy (C0 OBA) alongside Verification Transport Failure Rates under direct verification (V-LLM):
| Benchmark and Model | Native OBA (%) | Macro VTFR (%) | Delegation VTFR (%) | Compaction VTFR (%) |
|---|---|---|---|---|
| APB: Gemini-3.7-Flash | 64.8 | 18.3 | 17.8 | 11.7 |
| APB: GPT-OSS-120B | 48.0 | 23.4 | 26.9 | 17.4 |
| ATBench: Gemini-3.7-Flash | 56.9 | 10.9 | 10.6 | 4.4 |
| ATBench: Llama-4-Maverick | 56.2 | 25.2 | 29.9 | 19.5 |
| PRB: Gemini-3.7-Flash | 71.7 | 16.8 | 18.0 | 15.8 |
| PRB: GPT-OSS-120B | 71.5 | 32.7 | 27.6 | 22.9 |
| AgentDojo: Gemini-3.7-Flash | 76.5 | 6.4 | 6.2 | 5.4 |
| AgentDojo: GPT-OSS-120B | 75.0 | 15.1 | 17.0 | 13.8 |
The verification transport paradox across protocols
To test how verification methodology influences stability, the authors evaluated four verifier protocols: direct evaluation (V-LLM), evidence-grounded verification (V-EVIDENCE), agentic provenance-seeking verification (V-AGENTIC), and deliberative auditing (V-DELIBERATIVE). This comparison revealed a critical structural paradox: verifiers with lower aggregate transport failure rates are not necessarily more reliable.
Direct verification (V-LLM) registered the lowest aggregate VTFR at 16.4% under macro abstraction, 17.3% under delegation, and 14.1% under compaction. In comparison, V-EVIDENCE recorded 34.5% macro and 34.0% delegation failure rates, while V-DELIBERATIVE recorded 49.3% and 50.2%. However, failure-mode decomposition revealed that under macro abstraction, V-LLM produced 15.8% outright wrong verdicts and abstained on only 0.7% of cases. The single-pass verifier hallucinated confidence, issuing erroneous verdicts despite critical intermediary evidence being obscured.
Conversely, V-EVIDENCE produced only 10.6% incorrect verdicts alongside 24.0% abstentions, and V-DELIBERATIVE produced 9.4% incorrect verdicts alongside 39.9% abstentions. Grounded protocols recognized evidence limitations and appropriately abstained from speculative verdicts. As step-by-step verification methods inspired by Lightman et al. (2023) at arXiv:2305.20050 and process reward frameworks like ToolPRMBench (Li et al., 2026) at ACL Anthology become standard, distinguishing outright misjudgments from responsible abstentions is essential.
Provenance restoration as an evaluation safeguard
To test whether transport failures can be reversed, the study evaluated matched restoration conditions reintroducing the original execution trace. Across all benchmarks, models, and protocols, restoring execution provenance recovered 85.5% of transport failures (Restoration Recovery Rate, RRR), while causing only a 5.2% Restoration Degradation Rate (RDR) among judgments that had survived abstraction.
Recovery was highest for macro and delegation boundaries (87.9% and 86.9% RRR), compared to 79.8% under history compaction. Ablations on AgentProcessBench and ATBench confirmed that recovery depends strictly on provenance fidelity rather than re-evaluation alone. Simply repeating verification over the abstracted view (Requery) recovered only 22.5% to 33.1% of failures. Restoring lossy action identities without arguments or observations (LossyR) achieved 34.0% to 53.9% recovery, whereas restoring exact execution provenance (ExactR) recovered 73.6% to 82.5% of judgments.
Furthermore, varying abstraction severity (hiding 25%, 50%, or 75% of semantic evidence atoms) caused transport failures to increase monotonically, confirming that degradation stems from evidence starvation rather than the mere presence of an abstraction boundary.
What this means for research institutions and AI audits
The discovery that LLM verifiers degrade by approximately 31% across standard abstraction interfaces carries immediate practical consequences for universities, research administration, and frontier AI safety evaluation teams:
First, academic labs deploying autonomous agents for robotic experimentation, chemical synthesis, or computational pipelines cannot rely on high-level agent summaries for regulatory compliance or safety audits. If composite tools or multi-agent supervisor systems hide granular tool calls, automated verifiers may fail to catch invalid parameter ranges, data leakage, or protocol deviations. Research oversight bodies must require that agent environments maintain immutable, low-level execution logs.
Second, scientific data management standards must treat agent execution traces as primary research artifacts. Aligning execution traces with established metadata ontologies, such as the W3C provenance framework described in the CASRAI PROV-O term definition, allows institutional repositories to link generated findings back to specific tool calls and parameter states, preserving reproducibility and research integrity.
Third, funding initiatives and benchmarking organizations focused on autonomous safety, including grant programs supported by Schmidt Sciences AI safety initiatives and capability benchmarks structured around the METR time horizon evaluation framework, must incorporate multi-view verification into auditing protocols. A verifier that achieves high accuracy on flat benchmarks cannot be presumed reliable in modular agent stacks. Comprehensive auditing also requires registering model artifacts with persistent identifiers, as detailed in the CASRAI guide on persistent identifiers for AI models.
What happens next in agent verification standards
Following the release of the preprint on November 13, 2026, the work has been shared with the community ahead of the workshop on reliable agent development, Who Verifies the Agents? Toward Reliable Agent Development. The findings are accelerating discussions around how agent execution logs are archived, verified, and shared.
Benchmark developers are moving to incorporate multi-view evaluation suites that test verifier stability across abstraction shifts alongside standard terminal task metrics. Simultaneously, agent orchestration framework maintainers are investigating standardized provenance hooks that allow verifiers to request underlying execution evidence on demand. As autonomous agents assume greater responsibility in research workflows, formal verification transport standards will be critical to maintaining reliable automated oversight.
Sources
- Sheshanarayana, D., and Goli, S. L. R. (2026). Do Verifier Judgments Survive Abstraction? Measuring Verification Transport for LLM Agent Evaluation. arXiv preprint. Published November 13, 2026. https://arxiv.org/abs/2611.18471
- Debenedetti, E., Zhang, J., Balunovic, M., Beurer-Kellner, L., Fischer, M., and Tramèr, F. (2024). AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. NeurIPS / OpenReview. Published October 2024. https://openreview.net/forum?id=m1YYAQjO3w
- Fan, S., Ye, X., Huo, Y., Chen, Z.-Y., Guo, Y., Yang, S., et al. (2026). AgentProcessBench: Diagnosing Step-Level Process Failures in LLM Agents. arXiv preprint. Published March 2026. https://arxiv.org/abs/2603.14465
- Li, Y., Luo, H., Xie, Y., Fu, Y., Yang, Z., Shao, S., et al. (2026). ATBench: A Diverse and Realistic Benchmark for Tool-Use Agent Evaluation. arXiv preprint. Published April 2026. https://arxiv.org/abs/2604.02022
- Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., et al. (2023). Let us Verify Step by Step. arXiv preprint. Published May 2023. https://arxiv.org/abs/2305.20050
- Li, D., Yao, Y., Tan, Z., Liu, H., and Guo, R. (2026). ToolPRMBench: Evaluating and advancing process reward models for tool-using agents. Findings of ACL. Published August 2026. https://aclanthology.org/2026.findings-acl.602/







