Same 74 Tasks, Different People: Browser AI Agents Lose 36% of Their Success Rate

Computer-use agents, or CUAs, are AI programs that work in a browser the way a person does: they open websites, compare prices, make bookings and write emails. They are usually tested on fixed assignments: there is a wording, there is a checklist, and the agent either completes the task or it doesn't. A group of researchers from UT Dallas, USC, Carnegie Mellon, Georgia Tech, UCLA, the University of Pennsylvania, UC Irvine, the University of Illinois and MIT asked a different question: what happens when different people hand the same task to an agent, each in their own words?

The result is PersonaCUABench. The outcome is not encouraging: the average share of fulfilled requirements fell from 0.689 to 0.

441, a drop of 36%.

How One Task Became Four Different Requests

The tasks came from two existing benchmarks, so that results could be verified automatically:

• Odysseys: long information-gathering tasks on live websites, where the agent collects data, compares it and reports back. It contributed 38 tasks. • REAL: actions on exact replicas of popular websites, such as ordering food, sending an email or booking accommodation. It contributed 36 tasks. A program checks the outcome against the final state of the site.

That made 74 tasks in 11 domains: 40 from personal life and 34 from work. For each domain, the researchers created four personas, one per group: subject-matter experts, older adults, young adults, and practical users who live in deadline mode. That makes 44 personas in total. Each is described by 37 attributes from the MatrAIx schema of 1 290 parameters, including age, education, income, work experience, time pressure and familiarity with the topic.

A separate AI user simulator then took over. It received the profile and the original wording, and rewrote the request in the persona's voice. It could drop a requirement, describe it vaguely or add one of its own, for example demanding confirmation of a fixed price before a proposal is sent. The agent saw only the resulting dialogue and could ask clarifying questions. It had neither the profile nor the original task.

Nine models were tested: MiniMax M3, Kimi K3, Muse Glimmer 30B, Grok 4.6, Qwen 3.8 27B, Gemini 3.8 Flash, GPT-6 Astra, DeepSeek V4.1 Flash and Claude Opus 5.5. All ran in the same browser harness, with identical tools and the same step budget. That produced 3 330 runs. GPT-5.6-terra served as both the simulator and the judge; the judge was a separate instance and did not see the dialogue.

Where the Points Go

Every model's score dropped, with losses ranging from 28% to 45%. In all four persona groups, the score is lower than with the original wording. The best persona group trails the original by 0.185. Within a single model, the best and worst groups differ by 0.10–0.13. Across models, the spread on a single group reaches 0.14. In other words, who asks affects the outcome almost as much as which model does the work. The ranking shifts too: Gemini 3.8 Flash is near the top on the original requests (0.734), but lands seventh on one of the groups (0.360).

The authors broke each requirement down into three questions: did the user state it, did the agent attempt it, and was it fulfilled. Among the cases where points were lost:

• in 59.9%, the agent worked on what the person asked for but did it wrong; • in 11.2%, the user themselves asked the agent to stop or to confirm, and the check against the original checklist counted that as a failure.

The second point is telling. An agent that drafts a proposal but doesn't send it because the person asked it not to fails the test. In one example, MiniMax M3 stopped before sending; the task was formally failed, yet the user gave it 5 out of 5.

User Ratings Diverge from Test Scores

Average satisfaction across the persona groups was 3.59, 3.10, 3.42 and 3.33 on a five-point scale. For the original wordings, which have no profile, it is 4.04. The divergence runs in both directions.

• Among 646 fully completed persona tasks, only 62.7% received the top rating, and 25.9% gave a three or lower. For the original wordings, the figures are 84.1% and 9.5%. • Among 800 failed action tasks, users considered 56% done at the time of submission. After the site state was checked, 27% of them remained.

Clarifying questions don't save the agents either. They asked 609 questions, and only 24% of them concerned a requirement the user had not voiced. On GPT-6 Astra, the researchers tested two obvious remedies: showing the agent the user's profile and requiring it to ask at least one question. Neither one recovered the lost quality.

What This Means

For anyone building agents for email, shopping and bookings, the study is a warning: a high benchmark score does not mean real people will be satisfied. Where a mistake costs money or sends a message to the wrong recipient, you need checks that distinguish a deliberate stop at the user's request from a failure. You also need tests with different wordings of the same task. The authors propose evaluating agents not only on whether they completed the assignment, but also on whether they accounted for the requirements and constraints the person expressed.

The caveats are obvious: the users and the judge here are themselves AI, and from the same family at that, and there are only 74 tasks. But the direction is clear: a noticeable gap remains between "completed the task" and "helped the person."