LangGraph is a real, graph-native agent runtime with its own checkpointer for resuming and time-traveling through a run. Deep Agents and LangChain’s create_agent are built on top of it. LangSmith Deployment packages all of that with sandboxes, an auth proxy, tracing, and hosted infra. If you adopt that stack, you get a lot for free.
Kitaru is a different layer. It’s a replay-based eval layer that sits beside whatever harness you picked: kitaru-langgraph wraps your compiled graph — or your create_agent or Deep Agents build — and every invoke() lands as a session: model calls, tool calls, and graph callbacks recorded as nodes you can replay and score. It’s framework-agnostic by design — the same eval loop works whether the team next to you is on LangGraph, Pydantic AI, the OpenAI Agents SDK, Mastra, or the Vercel AI SDK.
Use Kitaru if you are
- A platform team whose app teams use multiple agent frameworks and want one eval loop across all of them
- Running on regulated or on-prem infrastructure where a hosted control plane is not acceptable — Kitaru self-hosts, Apache 2.0
- Turning the sessions that caught a failure into a cohort that gates the next change
- Deciding a model or prompt swap and want two runs over the same cohort compared side by side
- Already tracing to Langfuse, LangSmith, or Braintrust and want those traces runnable, not just readable
Use LangGraph / Deep Agents if you are
- Standardizing on LangGraph or Deep Agents as the single harness across teams
- Happy to adopt LangSmith Deployment as the packaged runtime + deployment story
- Resuming a crashed or interrupted graph from its own checkpointer — that's LangGraph's job, not Kitaru's
- Building graph-native agents where the graph abstraction is a feature, not an obstacle
LangGraph resumes the graph it was running. Kitaru replays the session to test what changes.
Graph-native vs framework-agnostic recording
LangGraph’s checkpointer, resume, and time-travel are powerful — inside the graph/state-machine model. Your agent is a graph of nodes and edges, and LangGraph persists state between supersteps.
Kitaru doesn’t require you to change the graph. KitaruGraphRunner wraps the compiled graph you already built — it doesn’t recompile it, replace the checkpointer, or change the result. invoke() and ainvoke() come back exactly as LangGraph returns them; the session is what’s new.
from kitaru_langgraph import KitaruGraphRunner
runner = KitaruGraphRunner(builder.compile(), agent_id=AGENT_ID)
result = runner.invoke({"request": task})One adapter, three ways in
LangChain’s create_agent and Deep Agents both return LangGraph runnables under the hood, so the same kitaru-langgraph adapter covers all three construction paths.
What changes is how much of the graph is replayable. A direct wrapper records the call and can replace the whole input. The two factory paths install middleware that also lets a replay override the model — with one live model call, never a cached response — and substitute matching tool results. Wrap a subagent opaquely and it still shows up in the outer session; it just doesn’t get its own substitution capabilities.
# Direct compiled graph
runner = KitaruGraphRunner(builder.compile(), agent_id=AGENT_ID)
# LangChain's create_agent — also a LangGraph runnable
from langchain.agents import create_agent
runner = KitaruGraphRunner.from_agent_factory(
create_agent,
factory_kwargs={"model": "openai:gpt-5.4-mini", "tools": [lookup_order]},
agent_id=AGENT_ID,
)
# Deep Agents — same adapter, optional extra
from deepagents import create_deep_agent
runner = KitaruGraphRunner.from_agent_factory(
create_deep_agent, factory_kwargs={...}, agent_id=AGENT_ID,
)Self-hosted eval server vs packaged platform
LangSmith Deployment packages runtime + sandboxes + auth proxy + tracing into a managed product. That’s genuinely useful if you want to offload the platform layer.
Kitaru ships the eval layer as a primitive you self-host: one FastAPI + Postgres server, with workers that replay and evaluate inside your own environment. It sits beside your observability stack rather than replacing it — Langfuse, LangSmith, and Braintrust stay your system of record; Kitaru is where you re-run what they recorded.
If your security team needs to know exactly where prompts, outputs, and traces live, “in our own systems” is a shorter conversation than a data-residency addendum on a hosted control plane.
Two different jobs: resume a graph vs replay a session
LangSmith ships real infrastructure for its job: LangGraph’s checkpointer persists state between supersteps so an interrupted or crashed graph can resume, and you can step back through recorded graph state. That durability lives inside the graph model, and it’s genuinely useful.
Kitaru is solving a different problem. KitaruGraphRunner wraps the compiled graph without recompiling it or replacing its checkpointer — it records one session per invoke() call, alongside whatever LangGraph is already doing. Depth depends on how the graph was built: a direct wrapper replays the whole input; create_agent or Deep Agents construction adds model and tool-result overrides on top.
Two different jobs, not competing answers: LangGraph resumes a crashed graph. Kitaru replays a session to test a change.
Compare runs, don't manage deployments
Both products have a real story here — aimed at different questions.
LangSmith Deployment exposes agent endpoints (MCP, A2A, Agent Protocol, HITL, memory APIs) as part of its packaged runtime.
Kitaru isn’t a deployment platform. An experiment runs the same cohort twice — once at a baseline, once with one variable moved — and client.compare(before, after) puts the two side by side. Drive it from the kitaru CLI, the Python SDK, the TypeScript SDK, or your coding agent.
What makes Kitaru unique
| Feature | Kitaru | LangGraph / Deep Agents |
|---|---|---|
| First-class graph/state-machine agent model | Not supported | Yes |
| Native checkpointer with resume and time-travel | Not supported | Yes |
| Packaged sandboxes + auth proxy + tracing (LangSmith Deployment) | Not supported | Yes |
| Hosted control plane option | Not supported | Yes |
| Every invoke() / ainvoke() call recorded as a replayable session | Yes | Not supported |
| Tool calls answered from the recording during replay | Yes | Not supported |
| Cohorts: frozen session sets as regression suites | Yes | Not supported |
| Experiments: same cohort, one variable moved, compared side by side | Yes | Not supported |
| Framework-agnostic adapters (Pydantic AI, LangGraph, OpenAI Agents SDK, Mastra, Vercel AI SDK) | Yes | Not supported |
| Self-hosted server, Apache 2.0 (FastAPI + Postgres, workers in your environment) | Yes | Not supported |
| Import Langfuse / LangSmith / Braintrust / OTel traces | Yes | Not supported |
How the two surfaces map
| Concept | LangGraph / Deep Agents | Kitaru |
|---|---|---|
| Layer | Graph-native agent harness + runtime | Replay-based evals (how you test what it did) |
| Core unit | A node, persisted by the checkpointer between supersteps | A session — recorded model, tool, and graph callbacks, replayable |
| Composition | Compiled graph, or create_agent / create_deep_agent | Same graph wrapped once by KitaruGraphRunner |
| Resume after a crash | Checkpointer + Command(resume=...) on your own thread | Not the job — Kitaru replays a session, it doesn’t resume a crashed graph |
| Replay depth | — | Direct wrapper: whole-input only. create_agent / Deep Agents: model and tool overrides too |
| “Did my change help?” | Read the traces, compare by eye | Two runs over the same cohort, diffed |
| Deployment / hosting | LangSmith Deployment (packaged runtime + auth proxy + tracing) | Self-hosted server; workers replay in your environment |
Code comparison
from langgraph.graph import StateGraph, END, START
from kitaru_langgraph import KitaruGraphRunner
builder = StateGraph(SupportState)
builder.add_node("normalize", normalize)
builder.add_edge(START, "normalize")
builder.add_edge("normalize", END)
runner = KitaruGraphRunner(builder.compile(), agent_id=AGENT_ID)
# Runs exactly as before — and lands as a session.
result = runner.invoke({"request": "Reset my password"})
# Later: freeze the sessions that matter, test a change.
import kitaru
client = kitaru.KitaruClient()
cohort = client.cohorts.create("hard-cases", sessions=session_ids)
experiment = client.experiments.create(
"cheap-model",
model="gpt-5-mini",
)
before = experiment.run(cohort=cohort, version="v1")
after = experiment.run(cohort=cohort, version="pr-311")
client.compare(before, after)from langgraph.graph import StateGraph, END, START
from langgraph.checkpoint.memory import MemorySaver
builder = StateGraph(SupportState)
builder.add_node("normalize", normalize)
builder.add_edge(START, "normalize")
builder.add_edge("normalize", END)
app = builder.compile(checkpointer=MemorySaver())
result = app.invoke({"request": "Reset my password"})
# One trace per invocation. The run that caught the bug
# is a transcript you can read — not a test you can run
# again with the model swapped.Pick the eval loop without picking the harness
If everyone on your team has standardized on LangGraph or Deep Agents and you’re moving onto LangSmith Deployment as the packaged runtime and platform, use what you have — Kitaru adds less there. If multiple harnesses live across your teams, or you want your production traffic turned into a regression suite you control, Kitaru gives you one self-hosted eval loop underneath whatever your teams already picked.
uv add "kitaru[cli,worker]" kitaru-pydantic-ai