Researchers from Ant International and Zhejiang University introduced Verify by Doing (VbD), an artificial intelligence framework that co-evolves task execution and active tool verification harnesses without updating model weights. Unveiled in November 2026, the architecture addresses a core bottleneck for research computing and data-intensive laboratories: enabling autonomous agents to self-improve and verify complex digital work when external ground truth or benchmark scores are absent.
What happened
In mid-November 2026, a research team from Ant International and Zhejiang University—led by Yitong Lin, Jianda Chen, Yihao Hu, Pan Wang, Hang Wang, Zhihao Wen, Xiaoxue Ren, and Wei Wu—published a technical study titled “Verify by Doing: Co-Evolving Agent Execution and Evaluation without External Feedback” (arXiv:2611.15276). The work addresses a persistent limitation in autonomous language agents: how to verify and improve complex multi-step task performance when external test-time feedback, grading rubrics, or reference answers are unavailable.
In real-world settings such as administrative workflows, software refactoring, scientific computing, and financial modeling, external reward signals do not exist at deployment. When agents rely on passive text inspection or superficial output checks—such as verifying file existence or matching table headers—critical functional defects remain undetected. Furthermore, generating additional solution candidates under flawed, static checks merely reinforces those blind spots, causing agents to accept regressions or drift into repetitive errors.
To overcome this bottleneck, the researchers introduced Verify by Doing (VbD). The system keeps the base foundation model weights fixed—evaluated primarily using DeepSeek-V4-Flash-0731—while co-evolving two separate software harnesses within a single task: an Actor harness that executes actions, and an Evaluator harness that actively inspects deliverables and environment states using programmatic tools.
Key details
Verify by Doing decouples execution from verification by establishing two distinct operational harnesses:
- Actor Harness: Governs direct execution through an editable set of instructions, modular skills, and operational checkpoints.
- Evaluator Harness: Directs active inspection of artifacts, database records, and environment state via tools, scripts, and queries.
Instead of relying on passive language-model critique, the Evaluator actively interrogates the environment. In a complex spreadsheet accounting task, for example, initial checks may only confirm that column headings exist. An updated Evaluator actively retrieves underlying cell values, recalculating expense totals and after-tax revenue offline to verify arithmetic consistency against task requirements.
To prevent erroneous updates from degrading the system, VbD implements decoupled selection and retention mechanisms:
- Evaluator Adoption: Candidate Evaluator harnesses are comparatively evaluated against the current Evaluator on a shared candidate pool. A new Evaluator is adopted only if its verification reports provide stronger evidential grounding and higher discrimination of flawed deliverables.
- Actor Retention: An updated Actor harness is accepted only when its fresh execution deliverable receives explicit validation from the active Evaluator.
- Output Selection: Choosing the final deliverable is decoupled from updating the Actor. The retained Evaluator independently selects the best deliverable from the historical pool, recovering successful results without generating new attempts.
The authors evaluated VbD across AutomationBench (covering Finance, Sales, Marketing, Operations, Support, and Human Resources), tau^3 Banking, and GDPval. Across all three benchmarks, VbD achieved the highest overall scores among methods operating without external feedback:
| Benchmark / Domain | Base Model (%) | Strongest Baseline (%) | VbD (Ours) (%) | Advantage (pp) |
|---|---|---|---|---|
| AutomationBench (Overall) | 22.3 | 33.7 (RQGM) | 37.0 | +3.3 |
| AutomationBench (Operations) | 42.3 | 48.3 (RQGM) | 56.3 | +8.0 |
| AutomationBench (Marketing) | 22.7 | 46.3 (RHI) | 52.0 | +5.7 |
| AutomationBench (Finance) | 35.3 | 52.7 (BoN) | 53.3 | +0.6 |
| tau^3 Banking Pass Rate | 35.1 | 45.4 (RHI) | 49.5 | +4.1 |
| GDPval Rubric Quality | 80.0 | 83.1 (RQGM) | 84.7 | +1.6 |
In addition to raw pass rates, VbD demonstrated an 89.3% True Negative Rate (TNR) on candidate regressions in AutomationBench, compared to 71.2% for Red Queen Gödel Machines (RQGM). Ablation tests confirmed that removing active tool verification reduced overall AutomationBench scores from 37.0% to 34.3%, while disabling Evaluator evolution reduced them to 35.4%, demonstrating that both active verification and evaluator adaptation are essential.
Background and context
The development of Verify by Doing addresses the growing focus on test-time compute in frontier language modeling. Early inference-time scaling strategies focused on best-of-n sampling, chain-of-thought majority voting, or self-consistency, where models sample multiple paths and rank them by internal confidence.
Subsequent frameworks introduced self-referential adaptation, including Handcrafted Gödel Machines (HGM), Reflexive Harness Optimization (RHO), and Red Queen Gödel Machines (RQGM). However, RQGM relied on held-out reference benchmarks or ground-truth scoreboards to evaluate new evaluator iterations. In real-world enterprise environments or customized scientific workflows, such scoreboards are unavailable.
Without external scoreboards, naive self-reflection suffers from evaluation drift: an evaluator sharing weights with the actor tends to overlook systematic mistakes. By requiring the Evaluator to collect empirical evidence through tools—such as database queries and offline recalculations—VbD grounds evaluation in the environment. This architecture mirrors professional research software engineering standards, where automated testing harnesses operate independently from implementation logic to maintain verification rigor.
Reaction and community analysis
The release of the preprint prompted widespread discussion across AI research and engineering circles. Technical commentary published by Grafy News on November 14, 2026, highlighted the cost efficiency of keeping foundation model weights frozen while evolving execution and verification routines. Grafy emphasized that VbD’s ability to recover successful deliverables from fixed candidate pools provides practical value for organizations seeking to maximize inference-time compute efficiency.
On technical discussion forums such as AlphaXiv and research tracking services like ArXivDaily, practitioners noted the significance of VbD’s 89.3% regression rejection rate. Analysts observed that this high true-negative discrimination mitigates the degradation commonly observed in recursive agent self-play, where flawed outputs gradually corrupt agent memory. Reviewers also commended the evaluation across realistic workplace domains rather than artificial puzzle benchmarks.
What it means for researchers, universities and research offices
The Verify by Doing framework offers tangible benefits for academic research offices, computational laboratories, and institutional IT administration:
- Auditable Research Administration: Research administration offices exploring AI for pre-award proposal routing, compliance screening, or expenditure tracking require transparent verification. VbD generates structured logs detailing which operational requirements are satisfied, violated, or unresolved, creating an explicit audit trail.
- Reproducible Scientific Workflows: Computational research groups frequently automate data wrangling, genomic pipelines, and statistical analyses. Incorporating active tool verification ensures that data pipelines maintain computational reproducibility. Research units can incorporate these principles into an institutional model evaluation suite that programmatically verifies intermediate files.
- Accessible Inference-Time Adaptation: Because VbD achieves performance gains without parameter fine-tuning, academic institutions facing compute constraints can deploy open-weight foundation models and rely on harness optimization to handle complex multi-step tasks.
- Infrastructure Demands: Implementing active verification requires institutions to invest in secure reproducibility infrastructure and automated workflows. Research computing units must support sandboxed environments where evaluators can safely inspect files and execute test scripts without risking shared systems.
What happens next
Following the mid-November 2026 publication of arXiv:2611.15276, the research team is expected to release open-source code repositories and modular harnesses for AutomationBench and tau^3 Banking. Academic groups are preparing to extend the framework into multi-modal domains, including graphical user interface automation and automated laboratory equipment operation.
Subsequent investigations are also expected to explore coupling VbD’s tool inspection with formal verification solvers, such as Satisfiability Modulo Theories (SMT) engines, to deliver mathematical guarantees for safety-critical academic and clinical applications.
Sources
- arXiv: Lin, Y., Chen, J., Hu, Y., Wang, P., Wang, H., Wen, Z., Ren, X., and Wu, W. (2026). Verify by Doing: Co-Evolving Agent Execution and Evaluation without External Feedback. arXiv:2611.15276 [cs.AI], November 11, 2026
- Grafy News: Verify by Doing lets AI agents self-improve without external feedback, November 14, 2026
- AlphaXiv: Community Research Overview and Discussion on arXiv:2611.15276, November 2026
- ArXivDaily: Research Paper Classification and Abstract for Verify by Doing (arXiv:2611.15276), November 2026







