Hallucination detector built from 6 language models caught errors in gold labels

A research team assembled a training-free hallucination detector from six off-the-shelf language models and achieved high metrics — but encountered an unexpected result: a group of examples where all LLM-judges unanimously considered the answer correct, despite the ground truth annotation marking a hallucination. When the authors ran their own verification procedure, it turned out that the detector was not the one in error, but the benchmark.

How the ensemble LLM-detector works

Instead of training a specialized model, the researchers combined six off-the-shelf language models in the role of independent "judges." Each model independently assesses whether the answer contains a hallucination, and the final verdict is rendered by consensus principle: if most judges agree, the result is considered reliable.

The training-free approach fundamentally lowers the barrier to entry: there is no need to collect a specialized hallucination dataset and no need to fine-tune the model for a specific domain. This makes the detector easily transferable between tasks. Metrics on standard benchmarks turned out to be high — which drew the authors' attention to a group of anomalous examples.

• Six independent LLM-judges without fine-tuning (zero-shot)
• Verdict by consensus of majority votes
• No need for specialized annotated data
• Easy transfer between domains and tasks
• High accuracy on standard benchmarks

Why judges disagreed with ground truth annotation

At first glance, the situation looked like a systemic blind spot of LLM-as-judge: the detector consistently "failed to notice" hallucinations where the ground truth marked them. Had this been confirmed, it would have been a serious vulnerability of the ensemble approach — meaning all six models share the same blind spot.

"This paper is less about the detector and not about the presumed

blind spot, but about the verification procedure that ultimately caught us ourselves," the authors write.

The researchers ran their own verification procedure: manually cross-checked disputed labels with primary sources. The result turned out to be completely opposite. Most "problematic" examples contained errors in the ground truth annotation — the answers there were indeed correct, and annotators or automatic labeling erroneously called them hallucinations. The LLM-judges had been right all along.

What this says about benchmark reliability

The problem of noise in annotated data (label noise) is well known in machine learning, but as applied to evaluating language models, it takes on special significance. Ground truth datasets for hallucinations are created by humans or semi-automatically; errors in them are inevitable, and it is precisely these that distort the final metrics.

The study revealed a paradoxical pattern: the higher the internal consistency of the detector, the more clearly it exposes errors in the benchmark itself. The unanimous consensus of six independent judges turned out to be a more reliable signal than a single annotation accepted as truth. This calls into question the standard procedure for evaluating LLM systems.

For practitioners, the conclusion is straightforward: before claiming a "blind spot" or systematic model error has been found, it's worth checking the quality of the ground truth itself. Annotation verification is not an optional step, but a mandatory part of the research procedure.

What this means

The research reminds us that trust in the "gold standard" of annotation is not unconditional — even if the dataset is widely used in the industry. For teams building LLM evaluation systems, this is a signal: implement a procedure for verifying the ground truth, especially where several independent judges consistently disagree with it. Sometimes the judges are right, not the annotation.