Multi-agent architectures are the most over-applied idea in enterprise AI right now. The pitch is appealing — specialists collaborating, each with a focused prompt — and the reality is usually one agent's worth of capability, three agents' worth of latency, and a debugging experience nobody enjoys twice.
That does not mean the idea is wrong. It means the bar is higher than 'it seems more organised'. After building several of these and deleting two of them, I use one test: does splitting reduce the context each model must reason over, or add genuine parallelism? If neither, it is one agent with extra hops.
The patterns that hold up
Supervisor with handoff is the workhorse. A router reads the request, picks a specialist, and hands over the conversation with a compact brief rather than the full history. The specialist has a small tool set and a focused prompt, which measurably improves tool selection accuracy — the failure mode of one giant agent with thirty tools is that it picks the wrong one, and the error rate climbs with the size of the menu.
Parallel fan-out earns its keep when the subtasks are genuinely independent: review this change for security, for performance, and for test coverage, then merge the findings. Wall-clock time is the win, and because the branches never see each other's output, there is no coordination cost.
Generator-critic is the third that consistently pays, but only with a real signal. A critic that re-reads the same context and offers an opinion mostly agrees with itself. A critic that runs the tests, executes the query, or checks the output against a schema is doing work the generator could not do, and that is where the quality actually comes from.
# State is explicit and typed. Every node reads and returns the same object,
# which is what makes a run reproducible and a failure debuggable.
class ReviewState(TypedDict):
diff: str
findings: Annotated[list[Finding], operator.add] # branches merge by append
verdict: str | None
graph = StateGraph(ReviewState)
graph.add_node('security', security_reviewer)
graph.add_node('performance', perf_reviewer)
graph.add_node('decide', adjudicator)
# Fan out in parallel, join at the adjudicator.
graph.add_edge(START, 'security')
graph.add_edge(START, 'performance')
graph.add_edge('security', 'decide')
graph.add_edge('performance', 'decide')
# The only loop in the system, with a hard bound.
graph.add_conditional_edges('decide', lambda s: 'security' if needs_recheck(s) else END)
app = graph.compile(checkpointer=checkpointer)Conversation is not a coordination protocol
The pattern that consistently disappoints is free-form agent conversation — a pool of agents talking until they converge. They converge on agreement, not on correctness, and the token cost is proportional to how politely they do it. Coordination should be code: an explicit graph with typed state, defined transitions, and a bounded number of iterations.
- Hand off a brief, not a transcript. Copying full history into every specialist is how you get cost with no capability.
- Type the shared state. Agents passing prose to each other lose fields silently.
- Bound every cycle. An unbounded critic loop is a budget leak with a plausible explanation.
- Checkpoint state so a failed run resumes instead of restarting, and so a human can inspect where it went wrong.
- Measure cost per resolved task, not per call. Multi-agent systems hide their cost in the number of calls.
If your agents need a meeting to reach a decision, you have modelled your org chart, not your problem.
Start with one agent and a good tool set. Split when you can point at the specific thing that is failing — context that does not fit, a tool menu too large to choose from reliably, or a genuinely parallel workload. Splitting for elegance gives you a distributed system where you used to have a function, and distributed systems charge rent.