Berkeley Describes Adaptive Parallel Reasoning: The Model Decides When to Branch

Reasoning models have run into simple arithmetic: the longer they think, the more expensive and slower they get. Berkeley AI Research's Stephen Xie and Tony Lian have released a survey of Adaptive Parallel Reasoning (APR). They argue that the decision to branch a line of thought should be made by the model itself, not by an external scaffold of heuristics. One of the authors, Tony Lian, co-led ThreadWeaver, one of the methods the survey covers, and discloses this explicitly in the text.

Why linear Chain-of-Thought became a bottleneck

Progress in recent years rests not only on more data and parameters, but also on scaling compute at inference time. Models that output intermediate steps, backtracking and hypothesis search dominate math, coding and agentic benchmarks. The price is twofold.

First, context. Accumulated failed branches dilute the model's attention, it gets worse at telling relevant material from distracting material, and quality drops. This effect is called context rot. Second, latency, which grows in proportion to the length of the reasoning. On planning tasks spanning millions of tokens, users wait tens of minutes and sometimes hours, according to the authors' citation of Qu et al.

Parallelism removes both problems: independent threads do not carry each other's context and run simultaneously. But most existing schemes impose structure on the model from the outside.

A catalog of approaches: from voting to a shared KV-cache

The survey sorts the field into three groups.

Simple fork-and-join:

• Self-consistency / Majority Voting (Wang et al., 2023): sample several complete chains independently and take the most frequent answer. • Best-of-N (Stiennon et al., 2022): the same, but a trained verifier picks the winner. • Shared drawback: the branches know nothing about each other, and compute is duplicated.

Heuristic structured search:

• Tree of Thoughts (Yao et al., 2023): BFS/DFS over "thoughts," with pruning via LLM evaluation. • Graph of Thoughts (Besta et al., 2024) and Skeleton of Thoughts (Ning et al., 2024): related decomposition schemes. • Monte-Carlo Tree Search (Xie et al., 2024; Zhang et al., 2024): nodes are scored by random rollouts, with a UCB-style balance between exploration and exploitation. • Drawback: you need to know in advance how to decompose the task.

Newer in-model variants:

• ParaThinker (Wen et al., 2025): two fixed stages, parallel generation of threads followed by synthesis. It uses trainable control tokens <think_i>, positional embeddings for each thought, and a two-phase attention mask. • GroupThink (Hsu et al., 2025): a single LLM runs several interdependent trajectories that see each other's partial progress at the token level and adjust on the fly during generation. • Hogwild! Inference (Rodionov et al., 2025): workers write to a shared KV-cache, and RoPE lets blocks of the cache be stitched together in different orders without recomputation. There is no explicit coordination protocol.

The common weakness of all three groups is that the decision of whether to parallelize at all is made on the model's behalf. The authors give an example: applying the same structure to "what is 25+42?" and to the problem of the smallest planar region in which a unit-length segment can be rotated 180° wastes compute on the first and probably picks the wrong decomposition on the second.

What adaptivity changes

Adaptivity in APR means dynamically allocating compute between parallel and sequential operations at inference time. The paradigm was introduced by Pan et al. in Learning Adaptive Parallel Reasoning with Language Models (2025). The survey's authors use the term "APR method" specifically for that concrete implementation.

In practice, the model generates special tokens that control the transition between modes. In the ThreadWeaver trajectory shown in the survey, a <Parallel> block contains two plans and two solution paths, after which the threads converge on a single boxed answer.

The authors name three advantages:

• Unlike Tree of Thoughts, it needs no domain-specific heuristics. Reinforcement learning yields general decomposition strategies. Models find patterns on their own, such as launching the next step in parallel with a self-check of the previous one, or hedging the main approach with a backup. • Unlike Best-of-N, there is no duplication. The model first forms a set of non-overlapping subtasks and only then hands them out to threads. • Unlike non-adaptive schemes, the model may not branch at all if the overhead of parallelism exceeds the benefit.

What engineers should check

For those building an inference stack, the survey raises three practical questions. First: measure wall-clock latency and total tokens separately. Parallel threads shorten the wait, but with poor decomposition the total cost may not go down. This conclusion follows from the logic of the approaches; it is not a figure from the survey. Second: thread isolation versus a shared KV-cache. Hogwild! saves recomputation and memory, while ParaThinker and the ThreadWeaver style give cleaner separation of contexts. Third: an adaptive model brings a new metric, namely the share of tasks where it correctly declined to parallelize. On simple queries, this refusal is what separates APR from Best-of-N with a fixed N.

For teams currently paying for long reasoning chains in agentic and coding scenarios, APR looks like the next step after simply making generation longer. Alongside the survey, BAIR has also formalized terminology that can now be used to compare such work.