Anthropic Details How to Build Evals for AI Agents: Three Grader Types and SWE-bench Going From 40% to 80%

Anthropic has published an engineering guide to evaluating AI agents, and its main point is down-to-earth: without evals, a team that has shipped to production starts fixing one thing and breaking another. An agent calls tools, changes the state of its environment, and adapts to intermediate results over many steps, so errors accumulate. Evaluating such a system takes a combination of methods. To illustrate, the authors cite figures: on SWE-bench Verified, models went from 40% to more than 80% in a single year, and Bolt built a working evaluation system in three months.

What an agent eval consists of

Anthropic starts with terminology, because the industry uses these terms inconsistently. A task is a single test with inputs and success criteria. Each attempt is called a trial, and trials are run several times because model output changes from run to run. A grader is logic that scores one aspect of the work; a single task can have several. A transcript is the full record of a trial: messages, tool calls, reasoning. The outcome is the final state of the environment.

The difference between the last two is fundamental. A booking agent may write "Your flight is booked," but the only thing that should count is whether a record exists in the SQL database. Another clarification concerns what is being measured: when you evaluate an agent, you evaluate the combination of the model and its harness, not the model alone. As an example, the authors describe a case with Opus 4.5 on the τ2-bench test. The model found a loophole in the flight booking rules and "failed" the task by the formal criteria, even though it came up with a better solution for the user.

Three types of graders

Agent evals are usually assembled from three kinds of checks:

• Code-based graders: string and regex checks, fail-to-pass tests, static analysis, verification of the final state and of tool calls. They are fast, cheap, and reproducible, but brittle against valid variations in the answer. • Model-based graders: rubrics, natural-language assertions, pairwise comparison, consensus among several judges. They are flexible and scalable, but non-deterministic, more expensive, and need calibration against humans. • Human graders: expert review, crowdsourcing, spot checks, A/B tests. This is the gold standard for quality, but it is slow and expensive.

The final score for a task can be weighted, binary (all graders must pass), or hybrid. The second fork is capability evals versus regression evals. Capability evals should start with a low pass rate and give the team a slope to climb. Regression evals should pass at nearly 100% and catch rollbacks. When a capability set saturates, its tasks "graduate" into the regression suite.

Coding and conversational agents

Deterministic checks come naturally for coding agents: the code runs, the tests pass. SWE-bench Verified gives the agent GitHub issues from popular Python repositories and counts a solution only if the previously failing tests are fixed and the existing ones are not broken. Terminal-Bench tests end-to-end technical tasks such as building the Linux kernel from source or training an ML model.

On top of tests, Anthropic advises evaluating the transcript: code quality heuristics and rubrics for agent behavior. An illustrative YAML file for the task "fix an authentication bypass" combines all the check types at once, including tests, ruff, mypy, bandit, and metrics for number of turns, tokens, and latency. In practice, the authors note, unit tests and an LLM rubric for code quality are enough.

Conversational agents (support, sales, coaching) differ in that the quality of the interaction itself is part of what is being evaluated. Here the evals rely on a verifiable end state and on rubrics that cover both task completion and the quality of the conversation.

Where to start

Anthropic's case for evals is practical. Early on, they force the team to pin down what counts as success: two engineers working from the same spec will understand edge-case behavior differently. Later, they provide baselines for latency, token usage, cost per task, and error rate. They also speed up the move to new models: teams with evals assess a model's strengths and weaknesses in days, while others test for weeks.

Examples from practice follow. Claude Code first evolved on user feedback, then gained evals for conciseness and file edits, and later for over-engineering of solutions. Descript evaluates its agent on three criteria: don't break things, do what was asked, do it well. The team moved from manual review to LLM graders with periodic human calibration. Bolt started late, when the agent was already in wide use.

The practical takeaway: don't wait for a perfect set. Take the tasks the agent currently gets wrong and turn them into a capability eval. Add code graders for the final state of the environment. Bring in an LLM judge only for what code cannot check, and verify it against human scores. Run each task several times, and move the ones that pass consistently into the regression suite.