On November 12, 2026, the Nova-Silicon research team published AnalogLayoutWorld, an open-source evaluation benchmark designed to test autonomous AI coding agents on physical analog integrated circuit layout tasks governed by strict geometric and electronic design automation constraints. Released as an arXiv preprint (arXiv:2611.16991v1), the study reveals that while modern frontier models reliably generate raw GDS layout files across 91.1% of trial runs, only 23.0% satisfy comprehensive acceptance criteria combining physical design rule checking, layout-versus-schematic verification, and post-layout electrical simulation.
Benchmarking Agentic Layout in Physical Analog Design
Automated digital chip design has long benefited from standardized logic synthesis and automated place-and-route toolchains. In contrast, analog circuit layout remains heavily dependent on human layout specialists because analog performance hinges on micro-level physical phenomena, including parasitic capacitance, device symmetry, thermal gradients, and electromigration reliability. To establish whether general-purpose language agents can navigate these physical constraints, the Nova-Silicon team—led by project lead Linyang Li and core contributors Peiji Li, Jianqiu Chen, and Wenjin Mai—constructed AnalogLayoutWorld as a rigorous synthetic benchmark comprising 40 expert-reviewed circuit layout challenges.
The benchmark tasks evaluate physical implementation starting from fixed transistor-level SPICE netlists targeting the open-source SkyWater 130 nm (SKY130) process design kit. Task families span common analog building blocks, including operational transconductance amplifiers (OTAs), bandgap voltage references, low-dropout regulators, data sampling circuits, and current-starved ring oscillators. Unlike traditional software benchmarks where evaluation relies solely on unit test scripts, analog layout requires grounding within a specialized computational environment capable of executing industry-standard electronic design automation (EDA) tooling.
Verification Toolchains and Iterative Feedback Loops
AnalogLayoutWorld provides autonomous agents with a containerized workspace built on the open Harbor framework. Within this sandbox, agents access open-source EDA utilities including gdsfactory for programmatic layout assembly, KLayout for physical Design Rule Checking (DRC) and Layout-Versus-Schematic (LVS) matching, Magic for parasitic extraction (PEX), and ngspice for post-layout transient and AC circuit simulations. Research groups deploying reproducibility infrastructure and containerized workflows will find the architecture notable: evaluation occurs in isolated containers with independent, trusted grading testbenches that prevent agents from tampering with verification rules or score scripts.
Each agent is allocated a three-hour wall-clock window per layout task. Successful execution demands iterative closed-loop reasoning. Agents must interpret multi-page DRC violation logs (such as minimum metal spacing and well-enclosure errors) and LVS netlist discrepancies, translate diagnostic feedback into spatial layout revisions, and regenerate GDS files. The benchmark evaluates performance through an objective multi-component scoring function: GDS generation (10%), LVS connectivity (10%), DRC rule conformance (10%), layout compactness (20%), engineering quality including common-centroid device matching and power routing (10%), and post-layout electrical simulation (40%).
Benchmark Results and Test-Time Scaling Limits
The researchers evaluated 13 distinct model-harness configurations across three independent runs per task, comparing frontier proprietary models, open-weight reasoning architectures, and Nova-Silicon’s specialized in-house analog layout system, Novara. The experimental data demonstrates wide divergence in physical reasoning and error recovery capabilities:
| Model / System | Execution Harness | DRC Pass (%) | LVS Pass (%) | Post-Sim Pass (%) | Full Acceptance (%) | Average Score |
|---|---|---|---|---|---|---|
| Novara Agent System | In-House / SGLang | 97.5% | 100.0% | 75.0% | 72.5% | 0.8154 |
| GPT-6 Astra [max] | Codex | 99.2% | 98.3% | 82.5% | 70.8% | 0.7483 |
| GPT-6.1 Sol [max] | Codex | 92.5% | 91.7% | 80.0% | 62.5% | 0.7023 |
| GLM-5.3 [max] | OpenCode | 80.0% | 69.2% | 42.5% | 35.0% | 0.4779 |
| Kimi-K3 [max] | Kimi Code | 50.0% | 40.0% | 30.0% | 20.0% | 0.2900 |
| Qwen3.8 Flash Next | Qwen Code | 40.0% | 41.7% | 25.0% | 14.2% | 0.2633 |
| DeepSeek V4.1 Flash | DeepSeek Harness | 37.5% | 30.0% | 17.5% | 9.2% | 0.2246 |
While Novara achieved the top aggregate score (0.8154) due to superior engineering quality and 100.0% LVS compliance, GPT-6 Astra led all general-purpose models with a 70.8% acceptance rate and 82.5% post-simulation pass rate. Notably, the study uncovered that 90.3% of successful DRC passes and 66.8% of LVS passes occurred after the initial verification call, confirming that iterative tool feedback is essential for progress. However, open-weight models suffered from budget saturation: GLM-5.3 Flash exhausted the three-hour time limit in 51 of 120 attempts because it consumed excessive tokens and tool calls on smaller circuits, leaving insufficient headroom to resolve complex layout defects on larger netlists.
Implications for Semiconductor and Hardware Research Groups
For university microelectronics laboratories and semiconductor engineering teams, AnalogLayoutWorld marks a transition from ad hoc prompt engineering toward standardized, reproducible AI experiments in hardware physical design. The benchmark underscores that generating syntactically valid layout code is largely solved, but closing the loop on geometric rules, device parasitics, and aspect-ratio bounds remains a formidable barrier for autonomous agents. Academic design groups exploring agentic electronic design automation must focus on test-time reasoning efficiency and structured tool interfaces to ensure that autonomous agents can reliably handle complex analog physical sign-off.
Sources
- Nova-Silicon Team. (2026). AnalogLayoutWorld: An Open Benchmark for Agentic Analog Layout Design. arXiv:2611.16991v1 [cs.AI]. https://arxiv.org/abs/2611.16991 (Checked 2026-11-15).
