AdaPRA lifts Qwen 80B by 24.5 points on data science leaderboards

Data science agents can run the entire machine learning cycle on their own, but they still trail humans by a wide margin on human leaderboards. A research team made up of Atharva Naik, Vivek Chari, Pol Llado, Daniel Fried, Carolyn Rosé, Benjamin Van Durme, Sam Thomson and Justin Svegliato has proposed AdaPRA, a system that targets the agent's work process itself. With five attempts per task, it raised the percentile of a Qwen model with 80B parameters by 24.5 points on the DSPredict-Easy benchmark and by 14.8 points on DSPredict-Hard, while also reducing the number of process errors. The preprint appeared on arXiv on 23 October 2026, and the authors have released the code for reproduction.

Where data science agents lose points

The authors start from an observation: the problem is not only that the model writes poor code. Agents make systematic mistakes in the course of the work itself. The researchers identify four types of failure:

• insufficient exploratory reconnaissance of the data;
• incorrect validation, which makes the quality estimate unreliable;
• ordinary bugs in the code;
• inefficient use of a fixed budget: in every new attempt, the agent reruns the same exploratory data analysis and the same baseline models.

The last item is especially telling. If an agent has five attempts per task and spends each one repeating the same work, the real budget for finding a strong solution shrinks several times over.

A persistent harness and an auditor agent

AdaPRA has two parts. The first is a persistent coding agent harness: it accumulates knowledge across attempts at the same task, so the results of exploration and baseline models do not have to be recomputed from scratch.

The second part is a process reward agent. Classic process reward models score reasoning steps. Here the auditor goes further: it examines the agent's trajectory and workspace, executes checks, and compares the work against rubrics. The rubrics are criteria for process quality, and they adapt to the specific task. After the audit, the agent receives instructions on which errors to fix.

A separate result concerns the rubrics themselves. Criteria seeded on easy tasks transfer to harder ones, and further adaptation of the rubrics yields an additional gain.

The numbers and transferability

• Base model: Qwen with 80B parameters.
• DSPredict-Easy: +24.5 percentile points on the leaderboard.
• DSPredict-Hard: +14.8 percentile points.
• Conditions: five attempts per task.
• Side effect: fewer process errors.
• Transfer across models: rubrics mined from runs of one base model improved results for DeepSeek-V3.2.
• Comparison with a competitor: AdaPRA outperformed DataPRM, a strong process reward model for data science.

The gain on the easy set is almost twice that on the hard one, which is an honest signal about the limits of the approach. Supervision fixes work discipline: checking, repetition, validation. It does not replace the model's ability to come up with a non-trivial solution to a hard problem.

What to take into practice

For teams building agents on open models, the main takeaway is practical. You can improve results without fine-tuning the weights: it is enough to add an audit layer with executable checks and memory across attempts. Rubrics accumulated from one model's runs work with another, which means they can be treated as a reusable asset rather than a one-off tweak for a particular model-and-harness pairing.

Those running agents on Kaggle-style tasks should check their own pipelines for repeated data exploration and weak validation: according to the authors, these are the errors that eat up the attempt budget. The open code makes it possible to compare AdaPRA with your own solution on the same DSPredict sets and see how much of the gap with humans process supervision alone can close.