The short version
- Split agents on permissions and tools, not on aesthetics. The honest trigger: one kind of question needs data another kind must never see.
- Routing should be the cheapest mechanism that works. Rules and regex route for free and fail predictably; an LLM supervisor adds a model call to every message and fails confidently.
- The case study: a support bot with exactly two agents — general product questions, and a refund specialist allowed to read ticket records — routed by regular expressions plus recent-history checks.
"Multi-agent" is the most oversold word in applied AI. In most diagrams it means a supervisor model that interviews the user, fans out to specialist models, and reconciles their outputs — a committee of LLMs. Occasionally that is the right design. Far more often, a product needs one agent, or two, with a boring router — and the committee version is slower, costlier, and harder to debug for zero extra capability.
This guide gives you the decision framework: what single and multi-agent actually mean, the signals that justify a split, how to route, and then the real version — how and why I split a production support bot into exactly two agents, and where I drew the line.
What the terms actually mean
A single-agent system is one prompt, one set of tools, one context-assembly path. Every user message gets the same treatment. Its strength is coherence: one place to fix behavior, one prompt to review, one path to test.
A multi-agent system is multiple specialist configurations — different prompts, different tools, different data access — plus a routing mechanism that decides which specialist handles each message. The specialists usually do not talk to each other; the router talks to them. (Systems where agents negotiate with each other exist, but almost no product needs one, and this guide will not pretend otherwise.)
The key insight: an "agent" boundary is really a permission boundary. Two agents that can see the same data and call the same tools are one agent with extra steps. The split earns its complexity only when the boundaries differ.
How you should decide
The three honest triggers
Work through these for your own product — the diagram below is interactive:
Should you split the agent? — answer honestly
- Do some question types need private data the others must never see?e.g. order records, billing, one user's account
- Do different tasks need different tools or side effects?e.g. one path only reads docs, another writes records or sends alerts
- Is one prompt accumulating contradicting rules?e.g. “be brief” for FAQs but “walk through every step” for billing
One agent, one prompt
Add retrieval and guardrails to the one agent you have. Revisit when a yes appears, not before.
Trigger 1: data isolation. Some questions need private records (orders, billing, one user's account); others must never touch them. In one shared prompt, every question pays for the record lookup, and every answer is one bug away from quoting someone else's data. A separate handler makes the permission visible in code, where a reviewer can check it — instead of buried in prompt wording, where nobody can.
Trigger 2: tool separation. One task only reads documentation; another writes records or triggers side effects like alerts. Giving every path every tool means your FAQ-answering path can technically send alerts. Nobody audits "technically."
Trigger 3: contradictory prompts. When one prompt says "be brief" for product questions and "walk through every step, never promise exceptions" for billing, the instructions fight. Specialists let each prompt be short, specific, and internally consistent — and short prompts are the ones models actually follow.
Zero triggers? Build one agent. Add retrieval, add guardrails, ship. Revisit when a trigger appears — splitting later is a refactor, not a rewrite, if you kept context assembly in one module.
How to route between specialists
Routing choices, cheapest first:
- Rules and regex. If the vocabulary that separates your domains is stable — and for money, cancellations, and account words it almost always is — pattern matching routes for free, in microseconds, with failure modes you can read in the code.
- Recent-history checks. Follow-ups rarely repeat keywords ("and if I just don't show up?"). Look at the last few assistant turns: if they were about the specialist's domain and the new message is not an obvious topic switch, stay with the specialist.
- A small classifier call. Only when wording is genuinely unbounded. Temperature 0, JSON output, a confidence floor, and a defined default when the call fails.
- An LLM supervisor agent. The expensive default in framework tutorials. It adds latency and cost to every message, and its failure mode — a confident misroute — is the hardest kind to debug. You need evidence the cheaper tiers miss real traffic before you pay this.
A useful discipline regardless of mechanism: stamp every reply with which agent produced it, and surface it in your logs or supervision channel. A misroute you can see is tuning; a misroute you cannot see is a mystery complaint.
How I implemented it on a real project
The support bot on the events platform I work on (introduced in the support-agent guide) has exactly two agents. The orchestrator is a map with two keys and a route function. If the route function returns nothing, general support runs. That is the entire framework.
Why the split happened at refunds
"Can I get my money back?" looks like every other support question until you try to answer it correctly. The answer depends on: is this person signed in? Which upcoming ticketed events do they hold? Did they name one? Did the organizer turn refunds off for that event? A knowledge base cannot know any of that, and a general agent that can read those records is trigger 1 violated on every message.
So:
- General support retrieves FAQ passages and reads the page context. It has no access to ticket records at all — not "instructed not to use them"; it has no code path to them.
- The refund-and-ticket specialist loads the user's upcoming ticketed events and the event's refund flag, resolves a deterministic state, and only then lets a model write sentences — with the state already decided in code. (The full state machine is in the hallucination guide.)
The router is regular expressions
The router lowercases the question and checks, in order: direct refund wording (refund, money back, cancel my ticket, can't attend plus money words — the same patterns the platform's dispute detector uses, so chat and alerts agree on the obvious cases); a few extra phrases (leave the event, refund policy, drop out); then follow-up context — the last six messages, assistant turns only. If recent turns discussed refunds and the new message is not an obvious new topic (create, invite, promote, analytics…), it stays with the specialist, so "yes, that one" does not fall back to a generic answer. Try the logic:
The router is rules — try it
The visitor asks…
General support
FAQs plus the page. Never reads the ticket list.
Refund & ticket
Reads this user's tickets and the event's refund flag before writing a word.
routed hereWhy: Regex hit: “money back” matches the refund pattern. No model needed to decide.
No model call anywhere in that decision. The misroute I accepted: a poetic, keyword-free refund request can land on general support. The mitigation is layered elsewhere — the platform's dispute-detection pipeline runs a small classifier on exactly the messages the regex missed (that tiering is its own guide), and a human can take over any thread. A chat misroute is recoverable; that is what makes the cheap router acceptable.
Where I drew the line
I would add a third agent the day a question type needs a fact the other two must not see — a payout-export specialist, say. I would not add an LLM supervisor on top of the regex router: an extra call on every message to replace a decision the regex already makes correctly for the traffic we actually see. And host-side actions (creating events from Claude or ChatGPT) are not a third support agent at all — they are a different system with confirmation gates, because they change data instead of talking about it.
Pitfalls to avoid
- Splitting by topic instead of by permission. "Billing agent, shipping agent, returns agent" with identical access is one agent cosplaying as three.
- Routing with a model because frameworks default to it. Price the supervisor: one extra model call per message, forever, plus a new confident failure mode.
- Letting specialists share a context assembler that injects everything. The specialist boundary is only real if the restricted data physically cannot enter the other prompt.
- No route visibility. Stamp the reply, log the route, read the logs.
FAQ
By the only definition that matters — separate permissions, separate prompts, a router — yes. The count is not the point; the boundary is.

Written by
Rushit Jivani
Senior Software Architect & AI Application Engineer. Architecture that shows up in crash rate, performance and how fast the team ships.