Multi-Agent Systems

When to Let Your Agents Work as a Team

Photo by Guoda Kavaliauskaitė on Unsplash

Somewhere along the way, most of us build the same thing. You start with one agent. It’s helpful, so you give it another tool. Then another. Soon it has a system prompt the length of a short novel, fifteen tools, and instructions for handling billing questions and writing code and booking travel and summarizing PDFs. It’s a Swiss Army knife that’s slightly bad at everything, and every time you add a capability, it gets a little worse at the others.

At some point you have a thought: what if, instead of one agent trying to do everything, I had several agents that each did one thing well? Congratulations - you’ve just reinvented the multi-agent system, and you’ve also signed up for a whole new category of problems.

Because here’s the thing that doesn’t seem so obvious at front: multiple agents are genuinely more powerful, and they are genuinely harder to get right. Coordinating a team is a different skill from doing the work yourself, whether you’re a manager of humans or an orchestrator of LLMs. So in this post I want to give you the honest picture: what multi-agent systems are, when they actually pay off (and when they very much don’t), the core architecture patterns you’ll reach for, how the two frameworks I work with day to day - Microsoft’s Agent Framework and Google’s ADK; express those patterns, and the hard-won lessons that’ll save you some pain. We’ll build real, runnable examples along the way.

What a multi-agent system actually is

Let’s pin down the term, because “agent” is one of the most overloaded words in the industry right now.

An agent is an LLM that autonomously uses tools in a loop to accomplish a goal — it reasons, decides to call a tool, looks at the result, reasons again, and keeps going until it’s done. A multi-agent system is simply several of these working together on a shared objective, usually with some structure governing who does what and how they communicate.

The mental model I like to think about: a single agent is an individual contributor. A multi-agent system is a team. And just like with people, forming a team doesn’t automatically make the work better. A well-organised team with clear roles outperforms a lone genius on big, parallelisable problems. A badly-organised team just generates meetings. The entire craft of multi-agent design is making sure you’re building the former.

That gives us our through-line for the whole post: more agents buy you more capability, but you pay for it in coordination. Every decision that follows is really about whether that trade is worth it.

When to go multi-agent — and when absolutely not to

This is the most important section in the post, so let me be blunt and honest: most problems do not need a multi-agent system. Reaching for one by default is the single most common mistake I see. So before any architecture, ask whether you should be here at all.

The case for going multi-agent is strongest when your problem has these properties:

1. The work is genuinely parallelisable. Independent sub-tasks that can run at the same time — researching ten companies, analysing a document from five angles — are the sweet spot.

2. It exceeds a single context window. When one agent would drown in tokens, splitting the work across agents with separate context windows adds real capacity. Each subagent compresses its slice down to just the essential findings.

3. Separation of concerns helps. Distinct tools, prompts, and instructions per role reduce interference and make each agent easier to reason about and improve.

4. The task is valuable enough to justify the cost. More on cost in a second — it’s steep.

How much does this actually help? Anthropic reported that their multi-agent research system — a Claude Opus 4 lead agent coordinating Claude Sonnet 4 subagents — outperformed a single-agent Claude Opus 4 by 90.2% on their internal research eval. That’s not a rounding error; for breadth-first problems, teams genuinely win.

But now the bill. Anthropic also found that agents use about 4× more tokens than a plain chat interaction, and multi-agent systems use about 15× more tokens than chat. Fifteen times. That’s the coordination tax made concrete, and it means multi-agent only makes economic sense for high-value tasks.

And there are whole classes of problems where multi-agent is the wrong tool, no matter how much you want to use it:

• Tightly-coupled work with lots of shared context. If every agent needs to see everything the others are doing, you don’t have a team — you have a very expensive way of playing telephone. Anthropic explicitly calls out that most coding tasks fall here: they involve fewer truly parallelisable subtasks than research, and today’s models still aren’t great at coordinating and delegating to each other in real time.

• Low-value or latency-sensitive tasks. If the task is cheap or needs to be instant, the token and coordination overhead isn’t worth it.

This is exactly the argument the team at Cognition made in their well-known piece “Don’t Build Multi-Agents” — when work can’t be cleanly split and context must be shared, a single coherent agent often beats a committee. It’s a healthy counterweight to the hype, and worth reading.

So here’s a decision heuristic before you write a line of code:

Decision heuristic for single vs multi-agent scenarios

If you land on “single agent” — good. You’ll ship faster and spend less. If you land on “multi-agent”, read on, because now the interesting design questions begin.

The anatomy of a multi-agent system

Before the patterns, let’s agree on vocabulary. Every multi-agent system, no matter how it’s arranged, is built from the same handful of parts:

• Agents & roles. The individual workers, each ideally with a narrow specialization — a researcher, a critic, a coder, a summarizer.

• Orchestration. The logic that decides who runs when. This is the heart of the system, and as we’ll see, it can be hard-coded by you (deterministic) or decided by an LLM at runtime (dynamic).

• Communication. How information moves between agents — either by passing messages to each other, or by reading and writing a shared state (a common scratchpad).

• Memory / state. What the system remembers within a run (and sometimes across runs).

• Handoff / delegation. The mechanism by which one agent transfers work or control to another.

• Termination. The condition that ends the whole thing — a final answer, a quality threshold, or a step budget so it can’t spin forever.

Here’s how those pieces fit together:

General architecture of a multi-agent system

Keep this picture in your head. Every pattern that follows is just a different way of wiring up these same components — different orchestration, different communication, different termination.

The architecture patterns

Now the fun part. There’s a small vocabulary of patterns that show up again and again, and once you can recognise them, most real systems are just combinations of these. I’ll give you each one, when to use it, and its main trade-off.

1. Sequential (pipeline)

The simplest pattern. Agents run in a fixed order, each consuming the previous one’s output — an assembly line. Draft → edit → fact-check → publish.

Sequential Orchestration

Use it when the task has genuinely ordered stages where each depends on the last.

Trade-off: zero parallelism, so latency is the sum of every stage; and an early mistake propagates down the whole chain.

2. Concurrent (parallel fan-out / fan-in)

Several agents work simultaneously on independent slices of the problem, and their results are merged by an aggregator. This is where multi-agent earns its keep.

Concurrent Orchestration

Use it when subtasks are independent — multi-source research, ensemble/voting, analysing something from several perspectives at once.

Trade-off: you need a real aggregation strategy (merge? vote? synthesise?), and independent agents can produce redundant or conflicting work.

3. Orchestrator-worker (hierarchical)

A lead agent analyses the request, decomposes it into subtasks, spawns worker/subagents to handle them (often in parallel), and synthesises their results. Crucially, the decomposition is dynamic — the orchestrator decides how many workers and what each does, at runtime. This is the pattern behind Anthropic’s Research system.

Hierarchical Orchestration

Use it when you can’t predict the steps in advance and need runtime decomposition of open-ended problems.

Trade-off: the orchestrator is a single point of failure and a potential bottleneck, and it lives or dies by the quality of its delegation — vague subtasks make subagents duplicate work or leave gaps.

4. Group chat (collaborative)

Multiple agents share one conversation and take turns “speaking”, usually coordinated by a manager/moderator that decides who goes next. Think of a panel discussion: a debate between a proponent and a skeptic, or a round-table of specialists reaching consensus.

Collaborative Orchestration

Use it when the value comes from multiple perspectives interacting — brainstorming, debate, red-teaming.

Trade-off: conversations can meander, repeat, or fail to converge, so you need firm turn-taking and termination rules.

5. Handoff (dynamic routing)

One agent handles the request until it decides another agent is better suited, then transfers control — passing along the context. The classic shape is a triage agent routing to specialists, or any agent escalating to a human.

Handoff Orchestration

Use it when different requests need different expertise and you want whoever is handling it now to decide the routing.

Trade-off: control-transfer needs clean context passing and clear boundaries, or the receiving agent starts cold and confused.

6. Reflection (loop / iterative refinement)

A generator produces something, a critic evaluates it, and the work loops back for revision — repeating until the critic is satisfied or a step budget runs out. It’s the agentic version of “sleep on it and edit tomorrow”.

Reflection Orchestration

Use it when quality matters more than latency and output can be meaningfully critiqued — writing, code generation, complex reasoning.

Trade-off: each loop costs time and tokens, so you must cap the iterations to avoid an agent polishing forever.

7. Magentic (dynamic planning)

The most sophisticated of the bunch, and worth knowing about even if you reach for it rarely. A manager agent maintains a live task ledger — a plan it writes, tracks, and rewrites as the work unfolds — coordinating specialist agents against open-ended problems. It generalises orchestrator-worker with explicit, revisable planning. Microsoft’s Magentic-One system is the reference implementation, and the Agent Framework ships this pattern directly.

Use it when the problem is complex and open-ended enough that even the plan needs to adapt mid-flight.

Trade-off: maximum power, maximum complexity — save it for problems that truly warrant it.

Picking a pattern

Here’s the cheat sheet:

Real systems mix these freely — an orchestrator-worker system whose workers each run a reflection loop, or a handoff that routes into a sequential pipeline. Learn the primitives, then compose.

How the frameworks express these patterns

Patterns are lovely in the abstract, but you build with a framework. The two I reach for — Microsoft’s Agent Framework and Google’s ADK — approach multi-agent from opposite ends, and understanding the difference tells you a lot about the design space. All the code below lives in my python-frameworks repo under ai/multi_agent/.

The key axis to hold in your head is who decides the control flow:

LLM driven delegation vs deterministic orchestration

Both frameworks can do both, but they lean different ways.

Google ADK: agents composing agents

ADK’s core idea is that agents are composable building blocks. It ships workflow agents — SequentialAgent, ParallelAgent, LoopAgent — that give you deterministic orchestration for free, and it lets a plain LlmAgent delegate to sub agents when you want the model to decide.

The concurrent + sequential pattern falls right out of composing two workflow agents. A ParallelAgent fans work out to specialists, then a SequentialAgent runs a synthesizer after them:

parallel_research = ParallelAgent(
    name="ParallelResearch",
    sub_agents=[market_researcher, tech_researcher, risk_researcher],
)

root_agent = SequentialAgent(
    name="ResearchTeam",
    sub_agents=[parallel_research, synthesizer],  # fan-out, THEN fan-in
)

How do the parallel researchers get their findings to the synthesizer? Through shared session state. Each researcher declares an output_key, and the synthesizer reads those keys back in its instruction — ADK’s answer to inter-agent communication:

market_researcher = LlmAgent(..., output_key="market_findings")

synthesizer = LlmAgent(
    ...,
    instruction="Combine the findings below...\n\n"
                "Market findings:\n{market_findings}\n\n"
                "Technology findings:\n{tech_findings}\n\n"
                "Risk findings:\n{risk_findings}",
)

The reflection loop is a LoopAgent. The crucial detail is termination — you get a hard max_iterations cap and an early exit when the critic calls a tool that sets escalate = True:

def exit_loop(tool_context: ToolContext) -> dict:
    tool_context.actions.escalate = True   # break out of the loop
    return {"status": "approved"}

root_agent = LoopAgent(
    name="WriterCriticLoop",
    sub_agents=[writer, critic],
    max_iterations=3,   # so it can never spin forever
)

And LLM-driven delegation is just an LlmAgent with sub_agents and an instruction to route rather than answer. ADK turns the model’s decision into a transfer_to_agent handoff:

root_agent = LlmAgent(
    name="SupportCoordinator",
    instruction="Do not answer yourself. Transfer to BillingAgent for billing "
                "topics, or TechnicalAgent for technical topics.",
    sub_agents=[billing_agent, technical_agent],
)

Microsoft Agent Framework: graphs and orchestrators

Agent Framework leans on an explicit graph of executors. At the low level you use a WorkflowBuilder and connect nodes with add_edge, including conditional edges that route on structured output — great when you want a precise, auditable flow. On top of that, it ships high-level orchestration builders for the common patterns: SequentialBuilder, ConcurrentBuilder, HandoffBuilder, GroupChatBuilder, and MagenticBuilder.

The concurrent pattern is a one-liner with ConcurrentBuilder, which wires the dispatcher → fan-out → participants → fan-in → aggregator graph for you. I passed a custom aggregator to merge the analysts’ takes:

workflow = (
    ConcurrentBuilder(participants=make_analysts())
    .with_aggregator(merge)
    .build()
)
result = await workflow.run("Should we launch a solar-powered edge camera line?")

The handoff pattern uses HandoffBuilder. You declare participants, a start agent, and which agents can hand off to which — the model then decides routing at runtime:

workflow = (
    HandoffBuilder(participants=[triage, billing, technical])
    .with_start_agent(triage)
    .add_handoff(triage, [billing, technical])
    .build()
)

Two philosophies, one table

Neither is “better”. ADK’s agents-composing-agents model is elegant for nesting and delegation; Agent Framework’s explicit graph is a gift when you need a precise, testable flow with conditional routing. Pick either based on your own experience of working with the two and personal preference of the implementation. You’d be able to do the same implementation in both the frameworks. The code would look quite similar too, so you really wouldn’t go wrong with either of them.

Communication, state, and the genuinely hard parts

Notice that the framework comparison ended on communication, because that’s where multi-agent systems actually live or die. There are two broad models, and you saw both above:

Shared state (a blackboard) — like ADK’s session state — is simple and transparent: everyone reads and writes a common scratchpad. It’s great for pipelines and fan-in, but it couples agents to a shared schema and can get messy at scale.

Message passing — Agent Framework’s edges — keeps agents decoupled and is easy to trace, but you have to be deliberate about what context travels along each edge.

That “what context travels” question is the crux of the hardest bug in multi-agent systems: the telephone game. Every time information passes between agents, detail can be lost or distorted. Pass too little context and the receiving agent starts cold and confused; pass too much and you drown it in irrelevance (and tokens). Getting this hand-off contract right is most of the work.

Two standards are emerging to tame the chaos, and they solve different problems. MCP (Model Context Protocol) standardises how an agent connects to tools and data — I wrote a whole post on MCP if you want the deep dive. A2A (Agent-to-Agent) standardises how agents talk to each other across frameworks and vendors. Roughly: MCP is how an agent reaches its tools; A2A is how agents reach their teammates.

Practical lessons and pitfalls

Everything above is architecture. Here’s the operational wisdom — most of it distilled from Anthropic’s write-up of running a multi-agent system in production, which I’d urge you to read in full.

1. Teach the orchestrator to delegate. The single biggest lever. Vague subtasks (”research the market”) cause subagents to duplicate work or leave gaps. Anthropic found each subagent needs a clear objective, an output format, tool guidance, and explicit task boundaries. Delegation is a skill you prompt for.

2. Scale effort to complexity. Agents are bad at judging how much effort a task deserves, so tell them. Anthropic embeds explicit rules — a simple fact needs one agent and a few tool calls; complex research warrants a whole team. Without this, agents spawn 50 subagents for a trivial query.

3. Guardrail against runaway behaviour. Cap iterations, cap subagent counts, cap tool calls. Multi-agent systems have emergent behaviour — small changes to the lead agent can unpredictably change how subagents act — so bound the blast radius.

4. Tool design is critical. An agent searching the web for something that only lives in Slack is doomed from the start. Each tool needs a distinct purpose and a crisp description; bad descriptions send agents down completely wrong paths.

5. Evaluating multi-agent systems is its own discipline. This connects straight back to my post on evals. You cannot assert a fixed sequence of steps, because two agents can take completely different valid paths to the same answer. Judge outcomes and whether the process was reasonable, not an exact trajectory. Start with ~20 real cases, lean on LLM-as-judge, and keep humans in the loop — they catch the emergent weirdness automated evals miss.

6. Observability is non-negotiable. When a run goes wrong across five agents and twenty tool calls, you need traces to find which step failed. In agentic systems minor changes cascade into large behavioural shifts, so tight feedback loops and durable checkpointing (so a long run can resume rather than restart) are what make them operable in production.

Wrapping up

Multi-agent systems are a genuine step up in capability — Anthropic’s 90.2% jump is real, and for breadth-first, parallelisable, high-value work, a well-designed team of agents will run circles around a lone one. But that power comes with a coordination tax measured in tokens, complexity, and emergent surprises. The skill isn’t wiring up agents; it’s knowing when the team is worth forming, choosing the right pattern, and getting the delegation and communication contracts right.

So my parting advice is the same as my opening warning:

Don’t reach for a committee when one good agent will do.

Start single. Move to multi-agent only when the problem genuinely pulls you there — then pick your pattern deliberately, instrument everything, and evaluate on outcomes.

Next, I want to go deeper on observability and tracing for agents — the missing piece that makes everything in this post debuggable when it inevitably misbehaves. If that sounds useful, subscribe to the compute blog with the button below so you don’t miss it.

Thank you so much for reading, and I’ll see you in the next one. Have a wonderful day!