Hallucinations in AI coding tools: why agents lose to find-and-replace

A CLI agent built on Qwen Code spent two hours on a task that an IDE's find-and-replace tool would have handled in 20 minutes. Constant hallucinations and unnecessary artifacts ate up the time.

When an agent loses to simple find-and-replace

The task looked trivial: make a set of repetitive edits across a codebase with a CLI agent. Qwen Code is a code-specialized model, and it was the one available in the developer's work environment instead of his preferred Claude. It should have finished quickly. Instead, the agent ran for about two hours. It added unnecessary constructs, lost track of the task, and needed constant manual correction.

The author kept trying to steer the model "out of sheer stubbornness by that point." That experience is what finally pushed him to dig into the nature of hallucinations instead of simply switching tools.

Key facts:

• Task: repetitive code edits via a CLI agent
• Expected time: 20 minutes with a manual tool
• Actual time: about 2 hours with the agent
• Model: Qwen Code
• Symptoms: hallucinations and extra artifacts

Why do "clever" prompts make hallucinations worse?

The article's central paradox: writing more detailed and "clever" prompts doesn't reduce hallucinations, and often increases them.

Context overload. A long prompt packed with conditions and exceptions makes it more likely the model loses the thread and starts filling in logic from its own weights rather than following the instructions.

Conflicting constraints. Detailed conditions can implicitly contradict each other. The model doesn't return an error. It picks one, picks wrong, and produces a plausible but incorrect result.

Out-of-distribution inputs. Code-specialized models were trained on a particular kind of request. Unusual or compound prompts push the model into territory with few training examples, and that's where hallucinations spike.

The illusion of understanding. A detailed instruction makes the developer feel the model has understood the task. In reality, the model is just generating the next token based on similar patterns in its training data.

What should developers do with AI agents?

The experience described in the article points to several practical takeaways:

• Atomize tasks: one agent call should handle one specific operation, not a chain of interdependent steps
• Check artifacts: verify the output of each step before passing it further down the pipeline
• Simplify, don't complicate: if the model hallucinates, try a shorter, more specific prompt
• Match the tool to the task: for repetitive mechanical edits, grep or a script is often more reliable than an agent

"Only after this wonderful experience did I finally decide to read the

manual," the author writes.

What this means

LLM hallucinations aren't a bug that a smarter prompt can beat. They're a systemic property of autoregressive models, and they get worse once inputs move outside the training distribution. Grasp that once, and you build AI pipelines around the tool's real limitations rather than in spite of them.

When is an AI coding agent less effective than

Find & Replace? When the agent hallucinates and generates incorrect code instead of simply applying repetitive edits. In the Qwen Code example, a task Find & Replace handles in 20 minutes took 2 hours because the agent kept adding unnecessary artifacts.

Why are AI agents sometimes worse than simple tools?

In the Qwen Code example, the agent added unnecessary artifacts and hallucinated instead of performing a repetitive code replacement, a task an IDE handles in 20 minutes.