Section 01 · Definition
What makes a system truly multiagent?
Quick answer
What is a multiagent system? A multiagent system is an architecture where two or more autonomous agents pursue a shared goal, each maintaining its own state, running its own reasoning loop, and communicating with other agents through defined protocols. The key word is autonomous: each agent decides its next action independently, based on its local state and the messages it receives. This is what separates a multiagent system from a single agent calling a sequence of tools.
The practical threshold: if you have one LLM reasoning loop dispatching to deterministic functions, you have a single agent with tools. If you have two or more reasoning loops that can independently decide, retry, branch, or escalate, you have a multiagent system. That distinction matters because multiagent systems require four layers of infrastructure that single agent systems can skip.
Why does this matter? Because the failure modes are different. A single agent with tools fails in predictable ways: the LLM hallucinates a tool call, the tool returns an error, the agent loops twice and gives up. A multiagent system fails in compounding ways: one agent's bad output becomes another agent's input, state diverges across parallel agents, and the orchestrator loses track of which agents have completed which tasks. These require a different architecture and four specific infrastructure layers.
Section 02 · Orchestration
Layer 1: The orchestration layer
Orchestration answers one question: who decides what each agent does next?
Supervisor agent pattern
A supervisor agent holds a model of the overall task, assigns subtasks to specialist agents, and decides what to do with each output. LangGraph formalizes this with its supervisor node. The risk is a single point of failure — if the supervisor enters a reasoning loop, the entire system stalls. Use supervisor patterns when tasks are inherently sequential and the number of specialist agents stays under eight.
Parallel dispatch
The orchestrator — deterministic code, not an LLM — fans out to multiple agents simultaneously and aggregates outputs. No LLM is in the critical path for routing. This suits tasks with clear decomposition: research pipelines, document processing, incident response. The trade off is rigidity: it cannot adapt if one agent's output changes what others need.
DAG dispatch
A directed acyclic graph defines agent dependencies: B runs after A, C and D run after B, E aggregates C and D. DAG dispatch combines parallel dispatch's explicit structure with partial ordering. The trade off is configuration overhead — workflow changes require DAG edits, not just prompt updates.
Practical decision rule: dynamic tasks requiring reasoning → supervisor. Known structure with maximal parallelism → parallel dispatch. Known structure with mixed sequential and parallel steps → DAG. When in doubt, start with supervisor and migrate to DAG when throughput becomes the constraint.
Section 03 · Memory and State
Layer 2: Memory and state management
State is where most multiagent systems break — not at the LLM layer, but at the state layer where data moves between agents and accumulates across turns.
Three types of memory matter. In context memory is the LLM call's context window: fast and consistent, but it disappears when the call ends and cannot be shared between agents in separate processes. External memory covers persistence layers outside the LLM call: databases, vector stores, KV stores, message queues. Episodic memory is a structured log of past actions and outcomes agents can query to avoid repeating failed approaches.
The state drift problem
State drift happens when two agents hold different beliefs about the same fact. Agent A reads a record and queues an update. Before it lands, Agent B reads the same record, makes a decision on the stale value, and queues its own update. One silently overwrites the other. Neither agent knows. This is almost certain for any multiagent system running agents in parallel against shared state without concurrency controls.
Optimistic locking
Each state record carries a version number. A write includes the version read. If the version has since advanced, the write is rejected and the agent must re-read before retrying.
Event sourcing
Agents emit events rather than writing state directly. A single reducer processes them in order and maintains authoritative state. All agents see the same ordered sequence of changes.
Partition by ownership
Each piece of state has exactly one agent that owns writes. Others read but never write. This eliminates the conflict at design time and requires no additional infrastructure. It is the right default.
Section 04 · Tool Integration
Layer 3: Tool integration and guardrails
Tools are how agents affect the world outside the LLM. In a multiagent system, multiple agents share a tool surface and the blast radius of a bad tool call multiplies.
Every tool call should be validated against a schema before execution. Without it, agents pass malformed arguments, tools return unexpected outputs, and the bad result propagates to the next agent. Enforce schema validation at the tool layer, not in prompts. Use Pydantic models for tool input schemas in Python systems. Reject invalid inputs with a structured error message the agent can use to revise the call, rather than raising an exception that aborts the run.
Tool call storms
A tool call storm happens when an agent retries a failed tool call without a hard cap, or when the orchestrator dispatches a failed agent again without checking whether the underlying tool is recoverable. The pattern: agent calls API, API returns 503, agent retries, 503 persists, each retry adds load to the degraded service and makes it worse.
Hard retry caps
Three retries with exponential backoff handles transient failures. Beyond three, the failure is likely structural and retrying makes it worse.
Circuit breakers at the tool layer
If a tool's error rate over the past 60 seconds exceeds a threshold, return an immediate failure to callers without hitting the downstream service. Close the circuit after a configurable timeout.
Concurrency limits per tool
A semaphore at the tool layer is simpler and more reliable than rate limit logic distributed individually across every agent that uses the tool.
Section 05 · Communication
Layer 4: Agent communication
How agents talk to each other determines how well they coordinate and how easily the system fails under load.
Message passing
Agents send messages through a queue or broker. Each consumes from its inbox, processes, and optionally sends to other agents. No shared state, no direct calls. This scales well: agents are decoupled, messages persist in the broker, and a failed agent can restart and continue. The trade off is latency — each interaction requires at least one queue write and read.
Shared state with ownership partitioning
Agents read from and write to a shared state store, but each piece of state has one owning agent. Faster than message passing for workflows where agents must frequently check each other's progress, but requires the drift mitigations from Layer 2.
Event bus patterns
An event bus sits between agents. Agents publish typed events and subscribe to the types they care about. This suits systems where coordination logic is complex and variable. The trade off is infrastructure complexity. Start with a simple queue (Redis Streams, SQS) and migrate to a full event bus only when routing logic genuinely requires it.
Pattern selection: start with message passing. Move to shared state if message latency becomes a bottleneck. Add an event bus only when the routing logic is too complex for either. For a deeper treatment of these patterns in the context of framework selection, see the production multiagent design patterns post.
Section 06 · Failure Modes
Three failure modes that sink production multiagent systems
| Failure mode | Root cause | Detection signal | Fix |
|---|---|---|---|
| Agent loops | No cycle detection in routing logic | Run duration exceeds expected maximum; token cost grows linearly | Add a step counter to agent state; route to error handler when step limit is exceeded |
| State drift | Concurrent writes to shared state without concurrency control | Two agents report conflicting facts about the same entity in the same run | Partition state ownership; one writer per state domain |
| Tool call storms | Unbounded retries hitting a degraded downstream service | Tool call volume spikes while success rate drops; downstream latency increases | Hard retry cap (3 max); circuit breaker at tool layer; concurrency limit per tool |
Agent loops happen when routing logic returns control to a node that has already executed, with state conditions that cause the same execution again. The fix is a step counter in agent state. Increment it after every node execution. If the count exceeds the workflow maximum, route to the error handler. Fail fast and predictably rather than running indefinitely.
State drift is the hardest failure mode to detect. Agent loops and tool call storms produce visible signals. State drift can be silent: both agents complete successfully, the workflow reports success, and the output is wrong. Detection requires consistency checks at aggregation points. When the supervisor or a final step collects outputs from multiple agents, compare them for internal consistency.
The circuit breaker pattern is the most effective fix for tool call storms because it decouples the retry decision from the individual agent. The individual agent sees one failing tool call. The circuit breaker has a view of that tool's error rate across the whole system and opens the circuit before the retry storm can compound. For more on deploying agentic AI to production with the observability infrastructure needed to catch these failure modes early, that guide covers the full production stack.
FAQ
Frequently asked questions
What is the difference between a multiagent system and a single agent system with tools?
A single agent with tools has one LLM reasoning loop that decides which tools to call. All decisions flow through that one loop. A multiagent system has two or more independent reasoning loops, each with its own state. The key distinction is independent decisions: in a multiagent system, each agent can decide to retry, branch, escalate, or stop based on its own state, without the central agent making that call. This independence is what makes multiagent systems more capable for complex tasks and more likely to fail in compounding ways if not architected carefully.
How do production multiagent systems handle agent failures and retries?
Each agent wraps its core logic in a retry handler with a hard maximum — typically three retries with exponential backoff — and returns a structured error state on exhaustion rather than raising an exception. The orchestration layer has explicit routing for that error state: a dedicated handler node that logs the failure, persists it to the checkpoint store, and returns a clean error to the caller. Retrying the entire workflow from scratch is rarely right; it is expensive and fails for the same reason.
What does a supervisor agent actually do in a LangGraph architecture?
A supervisor agent in LangGraph is a node that holds the task model for the overall workflow. It receives the initial task, reasons about which specialist should act next, dispatches to that agent via LangGraph's routing logic, and decides the next step after receiving the output. The supervisor returns a routing decision — not a final output — and LangGraph's conditional edges route to the appropriate specialist node. The supervisor does not execute tasks directly; it coordinates.
How many agents should a production system have?
Start with the minimum that cleanly separates the problem into independent responsibilities. Two to five agents covers most production use cases. Beyond five, coordination overhead frequently outweighs the specialization benefit. The right question is not how many agents but whether responsibilities genuinely benefit from independent state and reasoning, or are they just two steps in a sequence. If they are two steps in a sequence, implement them as two nodes in a single agent's graph.
If you are designing a multiagent system for production and need help with architecture decisions across these four layers, the agentic AI consulting service covers system design, failure mode auditing, and production readiness review.