Independent Critic Lifts Coding-Agent Accuracy by 5.9–9.5 pp

A research team that includes Zhengxin Zhang, Erkang Zhu, Yin Lin, Claire Cardie, Bolin Ding and Jingren Zhou tested the best way to organize verification in agent harnesses. They compared 17 schemes on seven top coding models and two benchmarks: Terminal-Bench (TB) and SWE-bench Verified (SWE). Independent review by a separate critic won. It added 5.9–9.5 pp of accuracy on TB and 3.9–6.6 pp on SWE, at 1.13–1.25x and 1.00–1.04x of the baseline run's cost, respectively.

Four layers where you can catch an agent red-handed

A harness gives a language model tools, shows it the results of using them, and lets it move toward the goal iteratively. The question is where in that loop to place verification. The authors split the agent lifecycle into four layers:

• Initial instructions. The model is asked to check itself in the starting prompt.
• Mid-run reminders. During execution, the agent gets nudges to double-check its work.
• Completion gates. The agent cannot declare a task done until it passes a checkpoint.
• Independent review. A separate critic examines the result, after which the agent makes fixes.

The first two layers turned out to be the cheapest and the most modest. The average gain on coding benchmarks is small and depends heavily on the model, but the number of output tokens does not grow noticeably. Completion gates squeeze out more, but they can raise costs substantially: the agent hits the gate and burns tokens on retries.

Why the critic wins on cost

Independent review landed at a favorable point on the accuracy–cost curve. On SWE the overhead stays within 0–4%; on TB it is 13–25%. For teams that pay for the tokens of thousands of agent runs a day, this difference between layers is easily measurable on the invoice.

Scaling works along two axes here. Additional review rounds or several independent critics add up to 3.5 pp more on TB over a single review. The result is test-time scaling achieved not through long reasoning chains from a single model but through a division of roles: the executor writes, and the critic reads without the baggage of the author's own assumptions.

Separately, the authors introduced VisualBench, a set for expert evaluation of visual and interactive artifacts, meaning interfaces and pages where tests catch far from everything. All four independent review schemes showed a positive net preference over the baseline, from 13.2 to 23.8 points. Each of them also outperformed the evaluated L0–L2 layer schemes.

Key figures

• Verification schemes compared: 17
• Models: 7 (frontier-level coding models)
• Benchmarks: Terminal-Bench, SWE-bench Verified, VisualBench (expert evaluation)
• Independent review on TB: +5.9–9.5 pp at 1.13–1.25x cost
• Independent review on SWE: +3.9–6.6 pp at 1.00–1.04x cost
• VisualBench: net preference of 13.2–23.8 points in favor of review
• Extra rounds or critics: up to +3.5 pp on TB over a single review

What to take into your own harness

If you are building a coding agent, cheap prompts like "check yourself" in the starting prompt are worth keeping, but don't expect a consistent effect: it depends on the specific model, so you need to run your own tests. Completion gates make sense where the cost of an error exceeds the cost of tokens and where you are ready to watch the budget.

The main practical takeaway is this. A separate critic pass after the work is finished, followed by fixes, delivers the most noticeable gain per unit of cost. It is best to start with one critic and one round. Add a second round or a second critic when, on your task, a single review stops finding errors. On visual and interactive tasks, where automated tests are weak, independent review is especially justified.

The paper was published on Hugging Face Daily Papers under arXiv number 2610.34577.