AgentOrch compares two types of agents: the gap between Global and Step-by-Step has narrowed by 79%

Enterprise agents typically run in a loop: act, observe the result, call the LLM again, take the next action. This is reliable but expensive, because the model runs at every step. The alternative is called global orchestration. The model builds a complete executable workflow once, and the system then runs it without going back to the LLM. Researchers led by Zijie Liu (nine authors) have released the AgentOrch benchmark and measured how far the second approach trails the first. Every evaluated model falls behind, but from early models to the strongest current one the gap has shrunk by about 79%.

What exactly was compared

AgentOrch is built on 650 enterprise standard operating procedures (SOPs) and 3220 runtime variants. A variant means that the same procedure meets different conditions during execution. For both modes, the authors fixed the same tools, environments and evaluation methodology. The difference in accuracy therefore cannot be explained by one agent getting a more convenient set of APIs.

The main result: step-by-step interaction (Step-by-Step) remains more accurate for every model tested. The reason is clear. An agent that sees the outcome of each action can correct course when the system returns an unexpected response or the data does not match expectations. A global workflow has no such option, so it notices an error in the plan only after execution.

Three ways to narrow the gap

The authors identified three directions, each of which improved global orchestration:

• Incremental workflow construction. Instead of generating the whole plan in one pass, the workflow is assembled in parts. Without execution feedback, this adds +9.98 percentage points of Global accuracy on average.
• Revising finished workflows with execution feedback. The plan is run first, then corrected based on the results. The method improved all evaluated models, including a gain of +6.13 percentage points on held-out runtime variants that the model had not seen during refinement.
• Targeted training on full workflows. Models are fine-tuned specifically on complete executable plans. The result: +12.18 percentage points on average across all five trained models.

These three paths work differently. The first changes how the workflow is generated, the second adds a verification loop, and the third changes the model's weights. The authors report figures for each separately, not for a combination. The paper therefore does not support the conclusion that the approaches stack, much less say what that would cost.

Why 79% matters more than the absolute gap

A 79% reduction in the gap says something about trajectory. Each new generation of models plans long action sequences better, and that improves Global almost for free. Step-by-step mode benefits less from model progress, because it was already compensating for weak planning with constant checks.

The gap is still not closed. The authors say so directly: Global offers a "promising path" to agents with higher accuracy and fewer LLM calls, but is not yet on par with step-by-step mode.

How to use this in practice

For teams building agents for internal processes (approvals, request handling, regulated operations), the results suggest several decisions.

• If a process is repetitive and well described in an SOP, it makes sense to try a global workflow: it cuts inference costs and reduces the number of points where the LLM can make mistakes or cause delays.
• If the environment is unstable and system responses often deviate from the norm, step-by-step mode remains the safer choice.
• Before choosing an architecture, it is worth testing workflow revision based on feedback. It improved all evaluated models and works even on unseen variants.
• If you have logs of successful executions, you can turn them into training data made of full workflows. On five models, this produced the largest average gain, +12.18 percentage points.

For the enterprise agent market, the takeaway is simple. The competition is not only about model quality. What matters more and more is how many times a model has to be called per business process. AgentOrch offers a way to measure that comparison under identical conditions.