The Multi-Agent Promise and Reality
Single-agent systems hit a ceiling quickly. When a task requires research, reasoning, code generation, and validation, a single LLM call chain becomes brittle and hard to debug. Multi-agent architectures solve this by decomposing work into specialized agents that collaborate — but they introduce new challenges around state management, error recovery, and observability.
LangGraph, built on top of LangChain, provides a graph-based execution framework that makes multi-agent systems manageable. Instead of ad-hoc chains, you define agents as nodes and their interactions as edges in a directed graph with built-in state persistence.
Core Concepts
LangGraph introduces three key primitives:
- StateGraph — A typed state object that flows through the graph and accumulates information as agents process it
- Nodes — Functions (often wrapping LLM calls) that read from and write to the state
- Edges — Conditional or unconditional transitions between nodes, supporting branching, loops, and parallel execution
This structure gives you explicit control over agent coordination instead of relying on an LLM to "figure out" which agent to call next.
Designing Your Agent Graph
Consider a customer support automation system with three agents: a Classifier, a Researcher, and a Responder.
from langgraph.graph import StateGraph, END
from typing import TypedDict, Literal
class SupportState(TypedDict):
customer_message: str
category: str
context: list[str]
draft_response: str
approved: bool
def classify_agent(state: SupportState) -> SupportState:
"""Classifies the incoming message into a support category."""
result = llm.invoke(
f"Classify this support request into one of: "
f"billing, technical, general.\n\n{state['customer_message']}"
)
return {"category": result.content.strip().lower()}
def research_agent(state: SupportState) -> SupportState:
"""Retrieves relevant documentation based on the category."""
docs = vector_store.similarity_search(
state["customer_message"],
filter={"category": state["category"]},
k=5
)
return {"context": [doc.page_content for doc in docs]}
def respond_agent(state: SupportState) -> SupportState:
"""Generates a response using the retrieved context."""
context_str = "\n".join(state["context"])
result = llm.invoke(
f"Using the following context, draft a helpful response "
f"to the customer.\n\nContext:\n{context_str}\n\n"
f"Customer message:\n{state['customer_message']}"
)
return {"draft_response": result.content}
# Build the graph
graph = StateGraph(SupportState)
graph.add_node("classify", classify_agent)
graph.add_node("research", research_agent)
graph.add_node("respond", respond_agent)
graph.set_entry_point("classify")
graph.add_edge("classify", "research")
graph.add_edge("research", "respond")
graph.add_edge("respond", END)
app = graph.compile()Adding Conditional Routing
Real workflows need branching. Add a human-in-the-loop approval step for sensitive categories:
def route_by_category(state: SupportState) -> Literal["research", "human_review"]:
if state["category"] == "billing":
return "human_review"
return "research"
graph.add_conditional_edges("classify", route_by_category)This ensures billing-related requests always get human review before a response is sent.
State Persistence and Recovery
Production systems crash. LangGraph supports checkpoint-based persistence using databases like PostgreSQL or Redis. When a node fails, you can resume from the last successful checkpoint instead of restarting the entire workflow:
from langgraph.checkpoint.postgres import PostgresSaver
checkpointer = PostgresSaver(connection_string="postgresql://...")
app = graph.compile(checkpointer=checkpointer)
# Resume a failed run
result = app.invoke(
initial_state,
config={"configurable": {"thread_id": "support-12345"}}
)This is critical for long-running agent workflows where a single failure shouldn't waste all prior computation.
Observability: Seeing Inside the Graph
Debugging multi-agent systems without observability is flying blind. Instrument your graph with LangSmith or OpenTelemetry to trace every node execution:
- Log input/output state at each node boundary
- Track LLM token usage per agent for cost attribution
- Measure latency per node to identify bottlenecks
- Record retry attempts and error rates per node
At MBB AI Studio, we build custom Grafana dashboards that show agent graph execution as a visual trace, making it trivial to identify which agent in a 10-step workflow caused a degradation.
Error Handling Patterns
Three patterns we use consistently:
1. Retry with backoff — Wrap LLM calls in retry logic with exponential backoff for transient API failures.
2. Fallback nodes — Define alternative nodes that activate when a primary node fails. For example, if the vector store is unavailable, fall back to a keyword search agent.
3. Circuit breakers — If an agent fails repeatedly, short-circuit the graph and route to a human operator rather than producing low-quality automated responses.
Testing Multi-Agent Systems
Testing agent systems requires a layered approach:
- Unit tests for individual node functions with mocked LLM responses
- Integration tests that run the full graph with a smaller, faster model
- Evaluation sets with known-good input/output pairs, scored by an LLM judge
- Chaos testing — randomly inject failures at node boundaries to verify recovery behavior
Conclusion
Multi-agent systems are not just a pattern for the ambitious — they're becoming the standard architecture for any AI application that goes beyond simple Q&A. LangGraph gives you the structure to build these systems reliably: typed state, explicit routing, persistent checkpoints, and full observability. Start with two or three agents, get the graph working end-to-end, then expand. The graph-based approach scales far better than the "one giant prompt" alternative.