Brains and AI knowledge bases: how to structure your documentation without causing hallucinations
You upload your company's manuals to an AI platform, connect an agent, and expect accurate answers. Instead, the agent contradicts itself, cites procedures that no longer apply, or invents facts that do not appear in any document. It is easy to think that the AI has a poor memory. Almost always, the problem lies in how the documents have been prepared.
In the previous chapter, you saw how to write robust system instructions. Now we focus on the other pillar of a reliable agent: the information it consults. You will see what happens behind the scenes when you upload a document to a Brain and how to prepare your documentation so the system retrieves it correctly. You will also find a protocol to keep the agent grounded in its sources and guidelines for organizing multiple knowledge bases.
The “document dump” problem
There is a recurring pattern in many companies. Everything available is uploaded to the platform: PDFs hundreds of pages long, old presentations, meeting minutes, and unreviewed spreadsheets. When the agent starts to fail, the technology gets the blame.
We call this document dump syndrome (data dump). It is not an AI failure, but a design failure.
As we saw in chapter 3, a model connected to a knowledge base follows the garbage in, garbage out principle: if garbage goes in, garbage comes out. RAG (retrieval-augmented generation) architecture does not fix ambiguous, disorganized, or contradictory documents. It simply delivers them to the model as they are.
For a knowledge base to work as your company's source of truth, you need to prepare the information before uploading it.
What happens when you upload a document to a Brain
To improve a knowledge base, it helps to first understand how it works. The process has four stages.
1. Chunking
The system divides each document into smaller blocks. Their size varies by platform, but usually ranges from a few hundred to around a thousand tokens.
If the document is poorly structured, a chunk may cut a sentence in half or separate a table from its title. That chunk loses its meaning.
2. Vectorization (embeddings)
Each chunk is converted into a vector: a list of numbers that represents its meaning. As we saw in chapter 1, this makes it possible to measure how similar two ideas are, even when they use different words.
3. Similarity-based retrieval
When you ask a question, the system also converts it into a vector. It then searches for the chunks whose meaning is closest to your question. Some platforms combine this search with keyword search.
4. Context injection
The retrieved chunks are added to the model's context window, alongside the system instruction. The model does not remember your documents. It reads only those chunks when it responds.
The practical consequence: if the retrieved chunks contradict each other or contain noise, the model works with incorrect premises. By noise, we mean broken headers, repeated footers, or scanning errors (OCR). The answer may sound confident and still be wrong.
Three principles for structuring a professional Brain
1. Convert your documents to Markdown
PDF is designed to look good on screen or on paper, not for a machine to interpret. It mixes columns, page numbers, repeated headers, and tables that do not extract well.
Best practice: convert your key manuals and procedures to Markdown (.md), a simple text format with a clear structure.
Follow a strict heading hierarchy:
markdown
This way, when the document is chunked, each paragraph retains its connection to the topic it belongs to. Also check that tables are preserved correctly and that headings stay with their content.
2. Make each section understandable on its own
Each section should make sense even when read in isolation. The system may retrieve a chunk without the text that precedes it.
Wording that is not useful for AI:
“In the case mentioned above, the fee will be 15 %.”
Wording that is useful for AI:
“For customers on Plan Empresa B2B (described in section 2), the annual renewal fee is 15 % of the base rate.”
Some guidelines to achieve this:
- Repeat the subject. Replace “this plan” or “that customer” with the specific name.
- Avoid vague references such as “the above,” “as already mentioned,” or “see above.”
- Define internal terms the first time they appear in each important section.
- One idea per section. If a section covers three topics, split it up.
3. Keep only one current version
If Politica_Devoluciones_2023.pdf and Politica_Devoluciones_2025.pdf coexist in the same Brain, the system will retrieve chunks from both because they are very similar. The model may mix incompatible rules in a single answer.
Basic rule: a Brain must contain only one current version of each document. Previous versions must be deleted or stored in an archive outside the operational knowledge base.
To keep things organized over time:
- Assign a person responsible for each document.
- Add the last review date and version number at the beginning.
- Review the Brain regularly and whenever a policy changes.
- Remove drafts, meeting minutes, and working documents that are not current rules.
A protocol to keep the agent grounded in its sources
A well-prepared knowledge base must be accompanied by system instructions that require the agent to base its answers on it. This is known as source grounding.
You can include a block like this in the system instruction:
This protocol makes the model work in extraction and synthesis mode. This leaves much less room to fill gaps with invented information.
Remember, though, that it greatly reduces the risk of hallucination but does not eliminate it. The model may misinterpret a chunk or cite the wrong section. That is why you should:
- Check citations in important answers.
- Test with questions whose answers you know and with questions the Brain cannot answer.
- Log which chunks were retrieved for each answer. As we saw when discussing high-risk systems, this traceability lets you reconstruct why the agent said what it said.
Specialized Brains instead of one giant Brain
Do not create a single knowledge base for the entire company. The more topics a Brain mixes, the more likely the system is to retrieve irrelevant chunks.
It is better to design knowledge bases by function, each connected to the agent that needs it:
- This improves accuracy because each agent searches only within its own area.
- It makes maintenance easier because each Brain has its own owners.
- It protects sensitive information because you can limit who accesses each knowledge base. For example, human resources data should not be accessible to the sales agent.
If your documents contain personal data, also keep the GDPR in mind: upload only the necessary data and review the platform's terms, as we saw when discussing data sovereignty.
Quick checklist for reviewing your Brain
Before connecting a knowledge base to an agent, check that:
- It contains only current, necessary documents.
- There is only one version of each policy or procedure.
- Key documents are in Markdown with clear headings.
- Each section is understandable without reading the previous ones.
- Tables retain their titles and are easy to read.
- Each document has an owner, date, and version.
- The system instruction requires the agent to cite sources and state what information is missing.
- You have tested the agent with questions the Brain can and cannot answer.
Conclusion
An agent's memory is only as good as the documents you give it. If you convert your manuals to a clear format, write self-contained sections, and keep only one current version of each document, the system retrieves information more effectively. If you also require the agent to cite its sources and connect it only to the knowledge base relevant to it, your documentation becomes a reliable source of truth.
Want to organize your documentation into knowledge bases that your agents can use reliably? Visit libertadia.com and sign up in the app.