Designing specialized AI agents: why dividing the work beats an “all-in-one” assistant
When a company starts using AI, it is tempting to create a single assistant that does everything: sell, write, review contracts, analyze numbers and code. It seems like the simplest option. But as soon as that assistant starts working on real cases, problems emerge: inconsistent responses, overlooked instructions and fixes that break what used to work.
In the previous chapters, you saw how to write robust system instructions and prepare reliable knowledge bases. Now we bring both pieces together to design agents. You will see why the generalist assistant fails, which principle to follow and which types of agents tend to work best. You will also find a complete practical example and the economic benefits of working this way.
The myth of the “god agent”
The most intuitive design is often also the least effective. An instruction thousands of words long asks a single model to act simultaneously as a sales advisor, persuasive writer, strict legal reviewer, financial analyst and programmer.
We call this approach the God Agent pattern. It tends to fail in production for three reasons.
1. Attention is spread too thin
As we saw in chapter 2, the attention mechanism determines which parts of the context carry more weight in each response. The more long, varied and sometimes contradictory instructions it has to handle, the easier it is for some to lose weight and for the model to follow them inconsistently.
2. The mindsets clash
A sales agent needs a friendly, empathetic tone. A risk auditor needs skepticism and rigor. If you ask the same agent for both, one will eventually override the other, or the result will land somewhere between the two.
3. Each fix can break something else
If the agent fails at an intermediate step, you have to modify the overall instruction. In doing so, you may unintentionally change its behavior on tasks that previously worked well. Maintaining it becomes increasingly difficult.
An idea with a long history
Specialization is not a new idea:
- In The Wealth of Nations (1776), Adam Smith used the example of a pin factory to explain how the division of labor multiplies productivity.
- In The Society of Mind (1986), Marvin Minsky proposed that intelligence emerges from the collaboration of many simple, specialized processes, not from a single central processor.
Something similar happens in applied AI. A set of small agents with well-defined functions and clear ways to pass information to one another tends to work better than a single generalist assistant.
Applying the single responsibility principle to agents
Software engineering has the Single Responsibility Principle, one of the SOLID principles popularized by Robert C. Martin. Adapted to AI, it states:
Each agent should have a single objective and a single reason to change.
To achieve this, define four dimensions for each agent:
Dimension
What it means
Example
1. A single function
The agent extracts, reviews or writes, but not all at once
“Extract data from invoices”
2. Its own Brain
Only the documentation it needs for its function
Billing manual, not the sales playbook
3. Only the necessary tools
Only the functions or connections it actually uses
PDF reading, with no email access
4. A strictly defined output
A fixed format in JSON or Markdown
A record with predefined fields
By removing secondary tasks from the agent, its entire context window is dedicated to the specific problem. You also reduce risks: an agent with no email access cannot send an email by mistake.
Four types of agents that work
Rather than creating one agent per department (“the marketing agent”, “the legal agent”), it is usually more effective to design agents according to the type of mental work they do. These four types cover most processes:
CLASSIFIER → EXTRACTOR → AUDITOR → WRITER
1. Classifier or router
- What it does: analyzes the input, identifies the type of case and decides which agent or workflow should handle it.
- Suitable model: a lightweight model (SLM) that is fast and inexpensive.
- What it delivers: a category code or a function call.
Example: it receives an email and classifies it as a “sales inquiry”, “technical issue” or “complaint”.
2. Extractor or researcher
- What it does: reads unstructured documents (invoices, contracts, emails, transcripts) and extracts only the key data in a standardized format.
- Key instruction: zero creativity and maximum fidelity to the document. If a piece of information is missing, it says so rather than inferring it.
Example output:
json
3. Auditor or critic
- What it does: does not create new content. It reviews the work of another agent or a person and looks for inconsistencies, breaches of internal policies or potential regulatory issues, such as those we saw when discussing the AI Act.
- Mindset: skeptical, methodical and focused on finding errors from the outset.
This is the “adversary” role we saw in chapter 5, now made a permanent part of the system.
4. Writer or formatter
- What it does: takes the validated data and the auditor's observations, and writes the final deliverable using your brand's tone, format and structure.
- Why it comes last: by this step, the data has already been checked. This lets it focus on communicating well, not on verification.
Practical example: preparing a sales proposal
Single-agent approach
You give an agent a customer's history and ask it:
“Read the case, check the manuals to see whether it fits the service, calculate the quote, assess the risks and write the sales proposal to send by email”.
Typical result: the quote contains calculation errors, the agent forgets to check an important risk policy, and the proposal is generic and unconvincing.
Specialized-agent approach
- Extractor agent: processes the customer's data and generates a structured record in JSON.
- Auditor agent, with the policy Brain: checks whether the record meets the requirements to qualify for the service. If information is missing, it stops the process and requests the missing data.
- Deterministic tool: calculates the exact price using a connected calculation function. The model does not do the arithmetic.
- Writer agent, with the sales playbook Brain: receives the validated record and the quote, and writes a proposal tailored to the customer.
- Human review: a person reviews the proposal and authorizes sending it.
A regulatory note: this case concerns whether a company is a fit for a commercial service, which is normally a minimal-risk use. If the same setup were used to assess individuals in areas such as employment, education, credit, or life and health insurance, it could fall into the high-risk category under the AI Act. Review the previous chapters before applying it in those cases.
Economic and operational benefits
Lower inference costs
You can use small, fast models for intermediate steps such as classification and extraction, which make up the majority of steps in many processes. More expensive, advanced models are reserved for the stages that really need them, such as critical review or complex writing.
Component-by-component maintenance
If the format of your proposals changes, you only update the writer agent's instruction. Extraction and review continue to work in the same way.
Errors are easier to locate
When a deliverable goes wrong, you can review each agent's output and see which step failed. If you also log the inputs and outputs of each step, as we saw when discussing traceability, diagnosis is much faster.
When not to divide the work further
Specialization also has costs. Each additional agent means more latency, more connection points and more instructions to maintain. Some guidelines:
- Do not split up a simple task that a single agent consistently handles well.
- Split the work when different mindsets are involved, such as creating versus critiquing, or when the required sources or tools change.
- Clearly define what the agents pass to one another. A clear output format at each step prevents errors from propagating through the chain.
- Start with a few agents and add more only when you identify a specific bottleneck.
Conclusion
An assistant that tries to do everything ends up doing everything inconsistently. If you divide the work among agents with a single function, their own knowledge base, only the necessary tools and a clear output, you get systems that are more accurate, cheaper and much easier to maintain. The key is not having the most powerful agent, but ensuring that each component does its part well and that a person oversees what matters.
Want to start designing your own specialized agents? Visit libertadia.com and sign up in the app.