Support triage agent that classifies tickets and drafts replies with LangChain and LangGraph

What they did

They started from the LangChain quickstart, using init_chat_model to bind a chat model behind a standard interface, then moved orchestration into LangGraph so the triage flow became an explicit state machine instead of a single prompt. The graph had nodes for intake normalization, intent classification, retrieval over their help-center docs, draft generation, and a confidence gate. Conditional edges sent low-confidence or high-risk tickets (billing disputes, security reports) to a human-review branch that paused the graph and resumed after an operator edit. Because LangChain's model abstraction is swappable, they A/B tested two providers on the same graph without rewriting node logic, and used structured output for the classification step so downstream routing read typed fields rather than parsing prose. They kept prompts and tool definitions in versioned files, added a small eval set of labeled historical tickets, and ran it before each prompt change. LangSmith tracing was wired in to inspect per-node latency and token spend.

How it went

The graph ran end to end on a few hundred historical tickets, with classification accuracy good enough to auto-route the clear majority and a review queue that caught the risky remainder. Swapping models took a config change rather than a refactor, which made the eval loop fast. Polish level was a working internal prototype: traces were readable, the human-in-the-loop pause/resume behaved correctly, and draft quality was acceptable for agent-assisted replies but not for unattended sending.

Worth knowing

Retrieval quality depended entirely on how clean the help-center docs were; stale articles produced confident wrong drafts. The confidence gate was a hand-tuned threshold, not a calibrated probability, so it needed periodic re-tuning. No production deployment, auth, or rate-limit handling was built, and the eval set was small enough that regressions could slip through. Cost and latency per ticket were only eyeballed from traces.

Wiring

Install langchain and langgraph (uv add langchain langgraph). Create the model with init_chat_model("openai:gpt-5.5") or another provider string, read from an env var so it is swappable. Define a TypedDict graph state holding ticket text, normalized fields, classification, retrieved docs, draft, confidence, and review status. Build the graph with StateGraph: add nodes for normalize, classify (using with_structured_output for a typed schema), retrieve (a retriever or tool call over the docs index), draft, gate, and human_review. Use add_conditional_edges after gate to branch on confidence and category. Compile with a checkpointer so the human_review interrupt can pause and resume. Wrap retrieval and any external lookups as LangChain tools so the model can call them. Add LangSmith tracing via env vars for per-node inspection, and a separate eval script that replays labeled tickets through the compiled graph.

Try it yourself

In this repo, build a support-ticket triage agent using LangChain for model access and LangGraph for orchestration. Requirements: (1) a graph with nodes for intake normalization, intent classification via structured output, doc retrieval, draft reply generation, and a confidence gate; (2) conditional edges that route low-confidence or high-risk categories (billing disputes, security reports) to a human-review node that interrupts the graph and resumes after an operator edit; (3) model selection through init_chat_model so the provider can be swapped by config alone; (4) prompts and tool definitions in versioned files; (5) a small labeled eval script over sample tickets that reports classification accuracy and draft pass rate; (6) optional LangSmith tracing behind an env var. Include a README with setup, env vars, and how to run the eval.