Multi-Agent AI Systems: How to Build Teams of AI That Actually Work
A single LLM call can answer a question. Multi-agent AI systems can research a topic, analyze the data, write a report, and send it to your team — autonomously. Therefore, the shift from single-prompt AI to coordinated agent teams represents the biggest practical advancement in AI engineering since ChatGPT launched. This guide shows you how to build these systems with LangGraph, including the patterns that work and the pitfalls that don’t.
Why Single Agents Hit a Wall
A single AI agent trying to handle complex tasks runs into three problems: context window limits (it forgets earlier steps), role confusion (it tries to be researcher and writer and reviewer simultaneously), and reliability (one bad step derails the entire output). Moreover, debugging a single agent that made a mistake on step 7 of a 12-step process is nearly impossible.
Multi-agent systems solve this by decomposing complex tasks into specialized roles. Each agent has a focused system prompt, limited tool access, and clear input/output contracts. Consequently, a research agent can search the web and compile facts, an analysis agent can identify patterns and insights, and a writer agent can produce the final output — each excelling at its specific role.
This mirrors how human teams work: you don’t ask your backend developer to also do UX research and write marketing copy. You have specialists who collaborate.
LangGraph: Building Agent Graphs
LangGraph models multi-agent workflows as directed graphs where nodes are agents or tools and edges define the flow of information. Unlike simple sequential chains, LangGraph supports conditional routing, parallel execution, and cycles (an agent can loop back for self-correction). Additionally, the built-in state management tracks everything that flows between agents.
from langgraph.graph import StateGraph, END
from langgraph.prebuilt import ToolNode
from typing import TypedDict, Annotated, Literal
import operator
class ResearchState(TypedDict):
"""Shared state that flows through the entire agent graph"""
messages: Annotated[list, operator.add]
research_data: str
sources: list[str]
analysis: str
quality_score: float
final_report: str
iteration_count: int
def research_agent(state: ResearchState) -> dict:
"""
Gathers information from multiple sources.
Has access to: web_search, document_reader tools
"""
research_prompt = f"""You are a research specialist. Your job is to gather
comprehensive, factual information about the topic.
Previous research (if any): {state.get('research_data', 'None')}
Quality feedback (if any): {state.get('analysis', 'None')}
Requirements:
- Find at least 5 distinct sources
- Include specific data points, statistics, and quotes
- Note conflicting information between sources
- Flag anything that seems unreliable
"""
result = research_llm.invoke([
SystemMessage(content=research_prompt),
*state["messages"]
])
# Extract sources from the research
sources = extract_urls(result.content)
return {
"research_data": result.content,
"sources": sources,
"messages": [AIMessage(content=f"Research complete: {len(sources)} sources found")]
}
def analysis_agent(state: ResearchState) -> dict:
"""
Evaluates research quality and identifies gaps.
No tool access — pure reasoning.
"""
result = analysis_llm.invoke([
SystemMessage(content="""You are a critical analysis expert. Evaluate the
research data for completeness, accuracy, and gaps.
Score the research 1-10 on:
- Source diversity (multiple perspectives?)
- Data specificity (concrete numbers or vague claims?)
- Completeness (any obvious gaps?)
If score < 7, explain what's missing so the researcher can try again."""),
HumanMessage(content=f"Research to evaluate:
{state['research_data']}")
])
score = extract_score(result.content)
return {
"analysis": result.content,
"quality_score": score,
"messages": [AIMessage(content=f"Analysis complete. Quality score: {score}/10")]
}
def quality_router(state: ResearchState) -> Literal["writer", "researcher"]:
"""Route back to research if quality is low, otherwise proceed to writing"""
if state["quality_score"] < 7 and state.get("iteration_count", 0) < 3:
return "researcher" # Loop back for better research
return "writer" # Quality is good enough, proceed
def writer_agent(state: ResearchState) -> dict:
"""Produces the final report from validated research."""
result = writer_llm.invoke([
SystemMessage(content="""Write a comprehensive, well-structured report.
Use the research data and analysis to create something genuinely useful.
Include specific data points and cite sources."""),
HumanMessage(content=f"Research:
{state['research_data']}
"
f"Analysis:
{state['analysis']}")
])
return {"final_report": result.content}
# Build the graph
workflow = StateGraph(ResearchState)
workflow.add_node("researcher", research_agent)
workflow.add_node("analyst", analysis_agent)
workflow.add_node("writer", writer_agent)
workflow.set_entry_point("researcher")
workflow.add_edge("researcher", "analyst")
workflow.add_conditional_edges("analyst", quality_router)
workflow.add_edge("writer", END)
# Compile with checkpointing for fault tolerance
from langgraph.checkpoint.sqlite import SqliteSaver
memory = SqliteSaver.from_conn_string("checkpoints.db")
app = workflow.compile(checkpointer=memory)
The Patterns That Actually Work in Production
Pattern 1: Supervisor + Workers. One agent (the supervisor) decides which worker agent to delegate to based on the task. This works well for customer support bots where the supervisor routes billing questions to a billing agent, technical issues to a tech agent, etc. The supervisor never does the actual work — it only routes.
Pattern 2: Pipeline with Quality Gates. Agents form a sequential pipeline (research → analysis → writing) with quality checks at each stage. If a stage fails the quality check, it loops back. This is the most reliable pattern for content generation and data processing workflows.
Pattern 3: Parallel Fan-Out / Fan-In. Multiple agents work simultaneously on different aspects of the same task, and a final agent combines their outputs. For example, analyzing a codebase: one agent reviews security, another reviews performance, another reviews code style — a final agent merges the reviews into a single report.
Pattern to avoid: Full mesh. Don’t let every agent talk to every other agent. This creates unpredictable behavior and makes debugging nearly impossible. Always have a clear information flow direction.
Multi-Agent AI Systems: Tool Isolation and Safety
Each agent should only have access to the tools it needs. A research agent gets web search and document reading. A database agent gets read-only SQL access. A deployment agent gets API access but only to staging. This isn’t just security — it prevents agents from taking shortcuts that produce incorrect results.
# Tool isolation per agent
research_tools = [web_search, document_reader, arxiv_search]
analysis_tools = [] # Pure reasoning — no tools needed
writer_tools = [grammar_checker, citation_formatter]
deploy_tools = [staging_api_client] # Never production access
# Each agent is created with its specific toolset
research_llm = ChatAnthropic(model="claude-sonnet-4-6-20250514").bind_tools(research_tools)
analysis_llm = ChatAnthropic(model="claude-sonnet-4-6-20250514") # No tools
writer_llm = ChatAnthropic(model="claude-sonnet-4-6-20250514").bind_tools(writer_tools)
State Management — The Hard Part Nobody Talks About
The most common failure in multi-agent systems isn’t bad prompts — it’s state management. When Agent A produces output that Agent B needs to interpret, the format must be consistent and parseable. However, LLMs are inherently non-deterministic, so Agent A might format its output differently each time.
The solution is typed state with validation. Use Pydantic models for state objects, validate at each transition, and include fallback parsing. Additionally, keep your state objects as flat as possible — deeply nested state creates debugging nightmares.
Checkpointing is equally critical. A 20-minute workflow that fails at minute 18 shouldn’t restart from scratch. LangGraph’s built-in checkpointing saves state after each node, allowing you to resume from any point.
Cost Management — Agents Can Get Expensive Fast
A quality-gate loop that runs 5 iterations with each iteration making 3 LLM calls means 15 API calls for a single task. At $3/million input tokens with Claude Sonnet, a complex research task might cost $0.50-2.00 per run. That adds up quickly.
Practical cost controls:
- Set maximum iteration limits (3 loops max for quality gates)
- Use cheaper models for routing/classification (Haiku for the supervisor, Sonnet for workers)
- Cache tool results — if two agents need the same web search, run it once
- Set per-request token budgets and abort gracefully when exceeded
Related Reading:
Resources:
In conclusion, multi-agent AI systems aren’t just a research concept — they’re the practical way to build AI applications that handle real complexity. Start with the pipeline pattern, add quality gates, and expand to more sophisticated topologies as your use cases demand it.