How Just AI Helped a Bank Break Through Its Automation Ceiling with LLM Agents

The Just AI team has published a breakdown of a real case: how to move a bank's customer support from NLU bots to LLM agents without getting a wave of hallucinations instead of higher automation.

A ceiling you can't drive around

Once an NLU scenario grows to several hundred branches, adding new dialogues stops helping. The bot starts making mistakes on edge cases, needs constant maintenance of its labeled data, and fails on unusual phrasing. The automation rate flatlines. This is exactly what the bank ran into: its mature NLU system for cashback hit its limit. The problem was not the quality of the tuning; it was the architecture. Classic NLU bots follow rigid rules: they recognize the intents they were trained on, but handle semantic variations and context within a long dialogue poorly.

What changed with LLM agents

Just AI proposed moving to an architecture built on LLM agents. Instead of a rigid scenario tree, a language model understands the meaning of the request, holds context, and generates an answer from the current knowledge base. Key changes in the system:

• The NLU classifier was replaced with LLM-based understanding, including paraphrases and unusual phrasing
• Support for multi-turn dialogues that preserve context
• Answers are built from the bank's knowledge base, not from hardcoded scripts
• A judge agent was introduced to check every answer before it is sent
• If an answer is not backed by a source, it is blocked and the customer is handed to a human operator

A judge agent against hallucinations

The main risk of deploying LLMs in bank support is hallucination: a model can confidently state wrong cashback terms or rules that do not exist. For a bank, this is not just bad UX; it carries regulatory and reputational risk. Just AI solved the problem with a two-tier check. The first agent generates the answer. The second, the judge agent, verifies it against the source knowledge base. The customer gets either a correct answer or a handoff to a live operator.

"The automation ceiling is not a bug, it is the architectural limit of NLU.

We helped the bank break through it by changing the technology, not by tweaking settings," — Just AI.

What this means

The move from NLU bots to LLM agents is not an upgrade but a change of paradigm. For banks and other regulated companies, it is only possible with built-in quality control: here the judge agent becomes a mandatory architectural element, not an option.