Architecture of an AI system: engine, fuel and mechanisms
When an AI tool gives an inaccurate answer or makes up a fact, many people think the model is not good enough. Others decide that AI is not yet useful for their sector. Often, the problem is not the model, but everything around it.
In this post, you will see which parts make up an AI system and the role each one plays. You will also see how to pinpoint the failure when the result is not what you expected, and how to combine the pieces to create reliable, scalable workflows.
The mistake of reducing AI to a chat
Many people and organisations evaluate artificial intelligence as if it were a single unit. To them, AI is a particular provider's chat window. If that window fails, everything fails.
This approach confuses a computing component with a complete system.
Think of a high-performance engine sitting on a workshop floor. It has plenty of power, but without a chassis, fuel, transmission or brakes, it cannot carry any load anywhere. Something similar happens with an isolated language model: on its own, it contributes little to a company's operations.
To build scalable digital assets, you need to distinguish between three components:
- The engine: the model that processes language.
- The fuel: the context and knowledge you give it.
- The mechanisms: the tools that allow it to act.
How the components work together
In a well-designed system, the pieces work as a loop:
code
The context feeds the model. The model decides and generates. The tools execute. A person oversees the critical points, and their corrections feed back into improving the context for the next cycle.
The engine: the inference layer
The engine is the base model, whether an LLM (large language model) or an SLM (small language model). It is important to be clear that it does not contain your private data or know your company's internal procedures.
What it does
- Processes natural language.
- Extracts and organises information.
- Converts content from one format to another.
- Analyses the relationship between parts of a text.
- Generates responses based on the material it receives.
How to choose it
There is no universal “best model”. The choice depends on the balance between four factors:
- Reasoning capability: how it solves tasks involving several logical steps.
- Latency: how long it takes to respond.
- Context window: how much information it can take into account at once.
- Cost per token: how much it costs to process the input and generate the output.
Its limitation
The model is not a store of facts or a database you can query. If you use it that way, sooner or later it will make up information.
The fuel: the context layer
The fuel is everything the model has in front of it while it works. It acts as its working memory.
What it includes
- System instructions: the assigned role, ethical and professional rules, and the required response format.
- Knowledge bases (Brains or RAG): technical documentation, procedure manuals, case law, records or customer data.
- Session history: what has been said in the conversation so far.
Why it is so important
The quality of the result depends largely on the quality of the fuel. If you give it disorganised, contradictory or irrelevant information, you reduce the performance of even the most advanced model.
This is the computing principle summed up as garbage in, garbage out: if garbage goes in, garbage comes out.
For example: if your knowledge base has three versions of the same procedure with no date or indication of which is current, the model may mix them in its response. Switching to a more powerful model will not fix it. Organising the documents will.
The mechanisms: the action layer
The mechanisms are the tools that connect the model to the outside world. They allow the system to look up information, execute tasks or take action in other applications.
Information retrieval tools
- Real-time web search.
- Relational database queries (SQL).
- Reading files and folders.
Execution tools
- Code execution (for example, Python or JavaScript) to perform exact calculations.
- Spreadsheet management.
- Document generation.
Operational integrations
- Sending emails.
- Updating statuses in a CRM.
- Creating calendar events.
- Triggering workflows in other tools.
How to diagnose where your AI system is failing
When a system produces a poor result, the usual reaction is to switch models. Before doing so, analyse which component is failing. This will help you find the real cause.
Symptom
Failing component
What to do
Makes up data or citations
Fuel: context is missing or RAG is not retrieving information properly
Provide the exact documentation in a knowledge base (Brain) and instruct the model not to answer beyond that source
Makes incorrect calculations
Mechanism: it is being asked to calculate by predicting text
Connect a code execution tool or a calculator through function calling
Does not follow complex instructions
Engine: its reasoning capability is insufficient for the task
Use a model with greater reasoning capability or break the task into simpler steps
Forgets instructions halfway through the process
Overloaded fuel: context degradation (context drift)
Clean up the context, summarise the history or distribute the process across several specialised agents
Notice that only one of the four symptoms is resolved by changing the engine. The others depend on the context or the tools.
Orchestration: from separate pieces to workflows
A system's power does not depend on having the largest model on the market. It depends on how you combine the pieces. That is orchestration.
A mature system usually combines four elements:
- Lightweight models (SLM) to classify and filter inputs at low cost with fast responses.
- Modular knowledge bases that provide only the information relevant to each step.
- Advanced models (LLM) that are used only in steps requiring synthesis or more demanding reasoning.
- Deterministic validations that check the data before moving to the next stage or requesting human review.
A practical example
Imagine a system that manages incoming enquiries at an advisory firm:
- An SLM classifies each enquiry by topic and urgency.
- The system retrieves only the internal documents on that topic.
- An LLM drafts a response based on those documents.
- A code tool checks the amounts and dates that appear.
- A team member reviews and approves the response before it is sent.
- An integration logs the enquiry in the CRM.
Each piece does what it does best. The result is more reliable, cheaper and easier to oversee than asking a single model to do everything in one conversation.
Conclusion
An AI system is much more than a model. The engine provides the ability to process language, the fuel gives it reliable knowledge, and the mechanisms allow it to act. When you understand this structure, you stop blaming the model for every failure and start making informed design decisions. This component-based architecture is what allows you to create scalable, reliable services with costs under control.
Want to start building your own AI systems? Visit libertadia.com and sign up in the app.