ABF-Bench: AI Agents Are Trained to Hunt Stolen Crypto, but They Quit Too Early

A team from Imperial College London (Xu Yu and William Knottenbelt) has introduced ABF-Bench, an interactive benchmark for AI agents that investigate the movement of stolen funds on a blockchain. The agent gets a few starting addresses and a limited budget of queries to network data, and it must reconstruct the path of the funds: where they went, and through which exchanges and bridges. According to the paper, $2.2 billion was stolen from crypto platforms in 2024 alone, and tracing such money is the foundation of anti-money-laundering (AML) controls and incident response. The benchmark has not been released yet: the authors promise to publish it after the paper is published.

An investigation you can verify without trusting the investigator

The central idea is that a public ledger records every movement of funds, so conclusions can be checked independently. In ABF-Bench, the agent submits a report made of "operations." Each operation contains the claimed transfers and links to the specific ledger records that support it. A deterministic verifier examines those records and issues a verdict without looking at the reference answer.

Confirmation rules are defined for each operation type. For example, a swap is proven by a transaction receipt and an execution trace, while a cross-chain hop is proven by a pair of records: a deposit and a payout.

If records are missing but nothing contradicts the claim, the operation stays "undetermined." A failed query does not count as proof that the transfer never happened.

There are several scores:

• Edge recall is the main metric: the share of edges in the hidden reference graph of fund movement that the agent found. An edge counts only if the agent itself loaded all the records it cited. • Fidelity is the share of submitted operations that passed verification. • Edge precision is the share of found edges that appear in the reference graph.

The reference graph is built conservatively: tracing stops wherever the records do not allow the path to continue unambiguously. As a result, every task has a definite answer, but the graph understates the real scale of the laundering. Alternative evidence is accepted if it fits the rules: a solution does not have to repeat the authors' path.

The tasks are built on real laundering schemes. They include multi-step transfers, DeFi swaps, circular routes between networks, deposits to custodial services and mixers, and address poisoning, where an attacker slips in a lookalike address. Some boundaries in a task are not disclosed: the agent must work out from an account's behavior that the funds have reached an unknown custodial service, and then stop.

Who took part and what came out

The authors compared three groups of solvers:

• ABF's own harness with working memory, in which the agent keeps a list of unverified branches. • General-purpose coding agents Codex and Claude Code. • Tracers with a fixed traversal order, including breadth-first search.

The ABF harness beat both general-purpose agents on final recall, including Codex running on the very same model. So the result depends not only on the model but also on how the agent is built.

Claude Code loses recall because of overconfidence: it cites records it never loaded. The verifier can resolve such references on its own, but under the benchmark's rules the edge is not counted in that case.

Early in the budget, the harness collects evidence faster than the best tracer. It also recognizes unlabeled custodial services that the tracers pass without noticing, at comparable fidelity.

However, all agents stop too early, especially on large tasks. As a result, the best final recall belongs to breadth-first search, not to the agents. The authors see this as room for improvement: the agent cannot yet tell in time that its work is not finished.

What this means for engineers and business

For teams building agents, ABF-Bench is useful because it measures verifiable behavior. An agent cannot reason eloquently without grounding in data: every claim is either backed by a record or it is not. The test also catches a typical agent failure: confident references to material the agent never read.

For compliance and AML teams, the benchmark shows that agents can already be trusted with manual routine work such as branching transfers and finding bridges, but cannot be relied on for the completeness of an investigation. For now, agents leave the tails unfinished, and those will have to be picked up by deterministic traversals or by people.

There is a second benefit. The verifier's verdicts can serve as a reinforcement learning signal, as is already done with coding tasks. Difficulty does not have to be invented by hand here: real attackers create it by changing their techniques from case to case.