Why We Threw Out 70% of a Corporate Knowledge Base Without Writing a Single Line of Code: The Engineering Anatomy of Enterprise RAG

• Share of problems caused by data: 70–80% of all enterprise RAG failures (30% outdated versions, 20% duplicates, 15% broken OCR)
• A real-world cleanup case: cutting the base from 12,000 to 3,800 documents produced a multi-fold jump in search accuracy
• Gain from Tool-based RAG: up to +70% answer quality when moving from blind pre-query retrieval to function calling
• Chunking parameters for regulations: Parent-Child (searchable child of 200–300 tokens, contextual parent of 1500–2000)

While building AI assistants and project knowledge bases at AlpinaGPT (the platform now serves over 8,000 active users and 40+ corporate customers), I've studied dozens of third-party RAG implementations. Almost everywhere, the same scenario repeats: engineers put together a slick prototype in a week, get the budget approved, and a month after launch the system is confidently citing orders that were cancelled three years ago and inventing contract clauses that don't exist.

Customers try to fix this by swapping out the LLM or reaching for trendy graph-based retrieval. But my experience shows a strict ratio: 80% of corporate tasks are handled perfectly well by classic advanced RAG with metadata, without the overcomplication of GraphRAG, while 70–80% of all failures lie in data engineering, not in the neural network's weights.

Why does corporate RAG hallucinate and surface outdated regulations?

Between a folder of corporate documents and a correct model answer lie four systemic bottlenecks:

1. A cluttered input corpus (Data Engineering): when a client hands us 10,000 corporate files, 30% turn out to be outdated editions with no dates, 20% are near-complete duplicates, and 15% are garbled PDF scans full of OCR noise and headers/footers. In one deployment, out of 12,000 files labeled "current," exactly 3,800 remained after an audit.

We simply deleted three-quarters of the file graveyard (2020 presentations and drafts), and output quality jumped multiple times over without a single line of code changed. The key check for week one: look at the share of files with a modification date older than one year. 2.

Vector search failures in Russian: OpenAI's standard `text-embedding-3-large` often stumbles on Russian industry terminology. According to our benchmarks, `multilingual-e5-large` and `jina-embeddings-v3` perform noticeably better on Cyrillic regulations. Moreover, plain vector search gets lost on queries like "clause 4.

2.1 of order No. 47" — here a hybrid loop of `vector + lexical BM25 search` with subsequent reranking is mandatory.

3. Silent degradation without monitoring: the knowledge base gets updated, and two months later accuracy drops. Beyond the basic trio of RAGAS metrics (`Faithfulness`, `Answer Relevancy`, `Context Recall`), version v0.

2 introduced a metric critical for enterprises — `Noise Sensitivity`, which measures how resilient an answer is to cluttered context. And the cheapest early-warning indicator of a RAG system breaking down in production is a weekly chart of the average answer length in tokens: as soon as the model stops finding the precise fact, it starts "padding with filler" and bloating paragraphs 2–3 weeks before the first user complaints arrive. 4.

A structural limit: RAG works with static snapshots and doesn't replace a transactional database (CRUD) for warehouse stock levels or live prices.

How should chunking for regulations, code, and FAQs be structured correctly?

Stuffing an 80-page PDF into a single vector, or splitting documents without preserving metadata (source, date, section, author), is a guaranteed way to kill search quality. There's no one-size-fits-all chunk size — we tailor the strategy to the physical nature of the document:

• Legal contracts and internal regulations: a `Parent-Child` strategy. Search runs against a narrow child fragment of 200–300 tokens (maximum retrieval precision), while a wide parent block of 1500–2000 tokens gets inserted into the model's prompt so that conditions and caveats aren't lost.
• Engineering documentation and API descriptions: a fixed `Fixed-size` window of 500 tokens with a 100-token `overlap`, so a thought doesn't get cut in half.
• Source code repositories: strictly semantic splitting along function and class boundaries.
• Short reference guides and FAQ bases: the `Sentence window` method — search by a single sentence, feed the model a window of `±N` neighboring sentences.

What should you actually do: why does

Tool-based RAG outperform native search by 70%?

In naive implementations, retrieval is hardwired before every model call (Native RAG): a user types "Hi, how are you?" into the chat, and the system dutifully pulls five random chunks of the company charter from the vector database, burning tokens and confusing the model.

We moved the architecture over to Tool-based RAG: the search engine is connected to the model as an external tool via `function calling`. The agent itself decides whether it needs to query the knowledge base, reformulates the user's question into a precise search query on its own, and, if needed, makes 2–3 targeted calls to different collections. In our projects, this single shift alone delivers up to a 70% boost in answer quality, and the two days spent writing a clean DOCX/PDF-to-structured-Markdown parser save months of fighting hallucinations. We cover the full kitchen of RAG design on pgvector and agentic tooling in depth in our engineering courses at [Hamidun Academy](https://academy.hamidun.com) and in architecture deep-dives at [Vibe Club](https://hamidun.com/club).