EvoWeb Tests Coding Agents on 320 Sequential Requests Across 32 Web Apps

Coding agents are usually evaluated the same way: give them a spec, get a finished app, run the tests. Real development works differently. A product grows through a chain of requests: add a cart, then promo codes, then a user account that has to show order history. The EvoWeb benchmark models exactly this process: 320 cumulative requests across 32 web apps, or an average of ten sequential steps per project. The authors report that the main bottleneck for current models is integrating new features with existing ones.

How the benchmark works

EvoWeb is built on real product features. The requests are linked by functional dependency graphs: one capability relies on another, so the order of tasks is not arbitrary. The agent works with a persistent codebase. The result of step 3 becomes the starting material for step 4, and accumulated errors do not go away.

The evaluation is unusual too. After each request, a multimodal web agent opens the intermediate version of the app and tests it in a live browser. It assesses three things:

• whether the newly requested functionality works;
• whether it integrates with features from previous requests;
• whether anything implemented earlier has broken.

The third check matters: in this setup, regressions do not hide behind a final test. They show up at every step.

Reliability instead of a stopping point

The authors made a separate design choice with the metric. EvoWeb has no fixed stopping criterion, such as "count until the agent fails a request." Instead, it integrates the trajectory over users' tolerance thresholds. One client will forgive a single failure; another will leave after the first. The metric shows how long an agent stays useful at different levels of strictness about errors. For an engineer, this is closer to the question "how many iterations can I trust it with" than a dry percentage of solved tasks.

What the experiments showed

The authors ran open and closed frontier models. EvoWeb clearly separated them by results. Many existing benchmarks struggle with exactly this, because the leaders bunch up in a narrow band. Cross-request integration proved to be the hardest component.

Two findings from the analysis:

• Requests that come later in the chain and have deeper or more numerous dependencies are solved worse.
• The number of runtime errors grows from request to request.

The second finding will be familiar to anyone who has trusted an agent with a long session. Code accumulates, links between modules grow more complex, and each new edit risks touching something the model has already "forgotten" or failed to keep in context. This is degradation over a long horizon, not an isolated miss.

How to use the results in practice

For teams adopting agents in development, EvoWeb offers a ready-made framework for their own evaluations. Assess the agent not on a single task but on a series of five to ten related changes in one repository, and always re-check old functionality after every step. Regression tests are not a luxury here but insurance. If runtime errors pile up, it makes sense to refactor more often and to shrink the context the agent has to hold in its head.

The signal for model and agent-framework developers is clear as well: improve how models handle dependencies and withstand accumulated complexity, not just the quality of generation from a blank slate. The benchmark's limitations are also worth keeping in mind: it focuses on web applications and relies on another AI agent as the judge, so its scores inherit that agent's margin of error. Still, as a testbed for long multi-request scenarios, EvoWeb fills a real gap.