How to design multi-agent systems? Comparing three architectures: Orchestrator-Worker, Hierarchical, and Peer-to-Peer

Six months ago we took over a stalled project. The client had a strong tech team and had built their own multi-agent system: one planning agent and six tool agents, all talking to each other in natural language. The demo looked beautiful. But two weeks into production, the same query would sometimes return in three seconds, sometimes forty seconds. Occasionally two agents would each call the same paid API independently and come back with contradictory answers, each unaware the other had done it.

The problem wasn't the model. It was the topology. They had designed agents to collaborate freely without any clear structure, and no one was responsible for the outcome.

The first design decision in a multi-agent system is never about which model to use. It's about choosing a collaboration topology. This piece covers the three patterns most common in production and the tradeoff criteria to help you select one.

Three basic topologies for multi-agent architecture

Multi-agent architecture design is a structural choice about who breaks down tasks, who executes them, and how information and control flow between agents. Topology choices affect system predictability, cost, and debugging difficulty significantly.

Orchestrator-Worker. A single orchestrator breaks down the task, dispatches it to stateless workers, collects results, and assembles the output. Workers don't talk directly to each other; everything flows through the center. The control flow is strictly unidirectional, so errors are straightforward to pinpoint.

Hierarchical. Sub-orchestrators sit under the primary orchestrator, each managing their own workers. It resembles an organizational structure. Use this pattern when a single orchestrator's context window can't hold all the subtasks, or when different domains need specialized prompts and tool sets.

Peer-to-Peer. No single authority. Agents talk to each other in parallel, challenge each other's reasoning, and negotiate toward conclusions. It's flexible in theory but difficult to control in practice. It loops, wastes tokens on redundant calls, and produces inconsistent outputs. The client we mentioned fell into this category.

Tradeoff comparison across three patterns

Enterprises often choose peer-to-peer because free-form dialogue sounds sophisticated. What they actually need is a system they can reproduce every time, track responsibility for, and predict costs on. That system is Orchestrator-Worker.

How to pick one: four selection criteria

Can you decompose the task upfront? If you can, use Orchestrator-Worker. Reviewing a contract, cross-checking three data sources, outputting a conclusion, these steps can be mapped in advance. You don't need agents negotiating in real time.

Move to Hierarchical when your orchestrator has to hold the intermediate state of many subtasks and prompts start bleeding into each other. Each sub-orchestrator owns just its slice. Partition lines usually follow business domains, not technical modules.

Ask who owns the failure. Production systems fail. You need to be able to point at a log line and say 'this is where the decision went wrong.' With mesh networks, when something breaks, there's no single point of accountability. Incident postmortems become impossible to trace.

Finally, can you forecast costs? Every round of peer-to-peer dialogue is another round of tokens, and the number of rounds isn't bounded. Unless the task genuinely needs multiple perspectives challenging each other, such as risk review or solution vetting, don't enable free-form agent conversation.

Start with Orchestrator-Worker, get it into production, and observe real traffic before moving to Hierarchical. Teams that start with peer-to-peer almost always spend three times the effort refactoring it back to centralized control after launch. Architecture evolves rather than being perfect from the start.

The client we mentioned needed only one change: consolidating their six parallel agents into a single orchestrator plus five stateless workers, and removing the natural-language interaction between agents. Latency variability disappeared. Contradictory answers stopped. Token cost per query became predictable.

Before designing a topology, answer these four criteria. A system running stably in production is what matters.