Claude Opus 5.5 scores 68% on HLE-Diamond, but each task costs 35 times more

The team at Artificial Analysis, an independent lab that compares AI models on quality and price, has measured what each extra point of accuracy costs. On HLE-Diamond, a set of very hard expert questions from the Humanity's Last Exam benchmark, Anthropic's Claude Opus 5.5 solves 42% of tasks at a low effort level and 68% at the maximum. That is a gain of 26 percentage points, but each task costs roughly 35 times more. OpenAI's GPT-6 Astra improved from 45% to 64%, a gain of 19 points, while its per-task cost rose 11 times.

What "effort" means here

Modern reasoning models do not answer right away. They first write a long draft of their reasoning: they try options, check themselves, and backtrack. The effort setting determines how much of this "thinking" is allowed. At a low level, the model answers quickly and briefly. At the maximum, it can spend many times more tokens on a single task.

Whoever calls the model pays for those tokens, so quality and cost rise together. Raising the effort gets you more correct answers, but you pay noticeably more for every attempt.

The numbers for both models side by side

The Artificial Analysis post gives the following data:

• Claude Opus 5.5: low effort 42%, maximum effort 68%, a gain of 26 points, per-task cost up roughly 35 times.
• GPT-6 Astra: low effort 45%, maximum effort 64%, a gain of 19 points, per-task cost up 11 times.

A few more figures follow from these. At minimum effort, GPT-6 Astra leads Opus by 3 points (45% vs 42%). At maximum effort the situation flips: Opus leads by 4 points (68% vs 64%). Opus's error rate fell from 58% to 32%, and GPT's from 55% to 36%.

This data does not show which model is cheaper. The figures only show how many times the per-task price grows within each model. In absolute dollars, Opus at maximum effort may cost either more or less than GPT-6 Astra at maximum effort. That depends on each model's starting price at the low level and on each vendor's token rates.

What each additional point costs

You can make a rough estimate of how price relates to quality. A 35-fold price increase equals about five doublings (2⁵ = 32). Over those, Opus gained 26 points, or about 5 points per doubling. An 11-fold increase equals about 3.5 doublings, and 19 points works out to about 5.5 points per doubling for GPT-6 Astra.

This is rough arithmetic. The real curve is almost certainly not a perfectly straight line, and gains should slow at higher levels. Still, the picture is useful: GPT-6 Astra is slightly more efficient at turning extra money into accuracy, while Opus 5.5 reaches a higher quality ceiling. Getting the last 10–15 points means paying disproportionately more.

What engineers and businesses should do with this

The main practical takeaway: "which model is better" tells you almost nothing without specifying the operating mode. The same model differs by 19–26 points across settings, which is more than the gap between the leaders at a single effort level. Compare "model plus setting" pairs, and compare them by cost per solved task.

Practical steps:

• For high-volume tasks such as classification, data extraction, and routine support replies, set low effort. Accuracy there rarely depends on reasoning depth, and the 35x markup will not pay off.
• Keep maximum effort for rare, high-value cases: complex analysis, scientific questions, hard bugs. An error there costs more than the extra tokens.
• Build routing. Start with a cheap pass, and send only the cases where the model is unsure or the result failed a check to the maximum mode.
• Measure cost per correct answer on your own data, not just the benchmark percentage. A test like HLE-Diamond shows the ceiling on exam-style questions, and your real workloads may behave differently.

The Artificial Analysis post is a good reminder that for reasoning models, accuracy has become an adjustable parameter rather than a fixed characteristic. The bill adjusts along with it.