Why we picked LangGraph over the Assistants API for production

The Assistants API is a great demo and a bad foundation. Three places it breaks under load, state ownership, tool-call concurrency, and cost routing, and what LangGraph (or equivalent) gives you instead.

This isn't a holy war about frameworks. The Assistants API is fine for prototypes and fine for low-traffic production. The failure modes are what matter, and whether you can absorb them.

State ownership

When a B2B tenant comes back six months later asking 'what did the agent see before it sent that email?', you need an answer. With the Assistants API, thread state lives in OpenAI's infra. You can fetch messages but not the full tool-input/output history with timestamps you'd want for an audit log. LangGraph keeps every node call in your database, in your schema, with your retention policy.

Tool-call concurrency

Run a load test on a multi-tenant Assistants API setup with ~30 concurrent users hitting tool calls. You'll watch p95 latency climb because tool calls serialize through OpenAI's runner. We hit this at engagement two. LangGraph runs in your process, concurrency is yours to tune.

Cost routing

Mid-graph model swapping is the difference between a sustainable inference bill and a runaway one. The Assistants API ties you to a single model per assistant; LangGraph lets you pick the model per node. We've seen 7×–10× cost reductions on shipped systems by doing nothing more than routing 'easy' nodes to Haiku-class models.