CrewAI and Kitaru solve different problems at different layers. CrewAI is an agent harness: model your agents as a crew of roles, give each one a task, and let crew.kickoff() coordinate the work. AMP wraps the whole thing in a managed platform on top. If your problem looks like teamwork and role-based delegation, CrewAI was built for it.
Kitaru is a replay-based eval layer: it records every run as a session, and a session re-executes. Unchanged, you get a faithful baseline. Forked with one thing changed, you get an answer.
There’s no first-class CrewAI adapter yet, so the way in is to import the traces you already collect from Langfuse, LangSmith, Braintrust, Logfire, or Arize Phoenix. An imported session inspects, investigates, and evaluates exactly like a recorded one. Want native recording and replay with overrides? The fastest route is a project-local adapter built with the kitaru-adapter-builder agent skill. It lives in your repo, not ours. Running that same crew durably in production is ZenML’s job; Kitaru replays and improves it. See ZenML vs CrewAI.
Kitaru
Use Kitaru if you are
- Running a CrewAI crew (or any agent) in production and want to test a change against real history instead of a vibe check
- Already tracing crew runs to Langfuse, LangSmith, Braintrust, Logfire, or Arize Phoenix and want those traces runnable, not just readable
- Turning the sessions that caught a failure into a cohort that gates every future change
- Comparing a model or prompt swap on the same frozen cohort before you ship it
- Self-hosting eval infrastructure so traces and credentials stay in your own systems
CrewAI
Use CrewAI if you are
- Modeling agent teams where roles, goals, and task delegation are the main abstraction
- Standardizing your whole toolchain inside CrewAI's model and AMP
- After a packaged managed platform with visual building, automations, and tracing
CrewAI designs how the crew collaborates. Kitaru proves whether a change to it actually helped.
Different questions, different layers
CrewAI is asking: how do I model a team of agents that collaborate on a task? Kitaru is asking: once that crew ships, which of its runs can I re-run? Import its traces, replay them, and see what a change actually did.
- Kitaru doesn’t compete for the harness layer. It has no opinion on roles, tasks, or delegation; that’s CrewAI’s job.
- Rolling out CrewAI means modeling your agents as a crew. Rolling out Kitaru means importing the traces the crew already produces, or writing a small project-local adapter against the recording API.
- The kickoff result, and the trace behind it, becomes a session you can investigate, evaluate, and replay. Overrides and tool substitution need an adapter in the process. An imported session on its own replays passthrough.
No adapter? Two honest ways in.
There’s no first-class CrewAI adapter today, and we’d rather say so than fake one. Two ways in, cheapest first. Import the traces you already collect from Langfuse, LangSmith, Braintrust, Logfire, or Arize Phoenix: no code, and an imported session inspects and evaluates exactly like a recorded one. Or build a project-local adapter against the recording API when you want native recording plus replay with overrides and tool policies. The kitaru-adapter-builder skill generates the smallest one that works and tells you what it can and can’t see.
kitaru session import sessions.jsonl --importer kitaru/langfuse@latest --agent crew@latest --wait, then investigate and evaluate like a native recording. Replay with overrides needs an adapter in the process.- Import.
kitaru session import crew-trace.jsonl --importer kitaru/langfuse@latest --agent crew@latest --wait: no code change, no adapter. Sessions land ready for investigation, cohorts, and evaluators. A replay of an imported session runs the crew again for real, with no overrides and no tool substitution. - Project-local adapter. Ordinary code that calls the recording API and lives in your repo. Wrap the public
crew.kickoff()entrypoint and change nothing else; a recording that changes behavior isn’t a recording. The same wrapper applies overrides and answers tool calls per policy during replay.
Self-hosted, on your own cloud
CrewAI AMP is a managed agent platform with a hosted control plane. If you want someone else to run the platform, that’s a fair trade. Kitaru takes the opposite one: one self-hosted FastAPI + Postgres server, workers that run replays and evaluations inside your environment, and sessions that never have to leave your systems.
- Self-hosted, Apache 2.0. One server, run it however you run services today.
- Workers in your environment. Replays, imports, and evaluations execute where you run them, so your traces and credentials don’t leave your systems.
- Beside your observability, not instead of it. Langfuse, LangSmith, Braintrust, Logfire, and Arize Phoenix remain where you watch production; Kitaru is where you re-run it.
Bring-your-own harness vs CrewAI abstractions
CrewAI asks you to model your agents inside its building blocks. Kitaru asks for a trace. The harness under a session can be CrewAI, Pydantic AI, the OpenAI Agents SDK, or hand-written Python. Kitaru cares whether it can see a recording, not how you produced it.
from crewai import Agent, Task, Crew researcher = Agent(role="Researcher", ...) writer = Agent(role="Writer", ...) research = Task(agent=researcher, ...) draft = Task(agent=writer, ...) crew = Crew(agents=[researcher, writer], tasks=[research, draft]) crew.kickoff(inputs={"topic": "agents"})
# No CrewAI adapter yet: import the trace kitaru session import crew-trace.jsonl \ --importer kitaru/langfuse@latest --agent crew@latest # A Pydantic AI agent: native adapter, one wrapper from kitaru_pydantic_ai import KitaruAgent support = KitaruAgent(agent, agent_id=AGENT_ID) support.run_sync(text)
- Native or imported. Pydantic AI, the OpenAI Agents SDK, LangGraph, the Claude Agent SDK, Mastra, and the Vercel AI SDK have a first-class adapter. CrewAI doesn’t yet: import its traces, or build a project-local adapter against the recording API.
- No graph DSL. Whatever the harness decides (task delegation, tool calls, conditional branches), Kitaru records the outcome instead of re-implementing the decision logic.
- Different harnesses, one eval layer. When different teams pick different harnesses, one place to record and replay is a smaller ask than one harness across the org.
What makes Kitaru unique
| Feature | Kitaru | CrewAI | What that means |
|---|---|---|---|
| Multi-agent abstractions (Agents, Tasks, Crews, Flows) with role-based delegation | Not supported | Yes | |
| Tools, knowledge/RAG, MCP servers, and an integrations marketplace | Not supported | Yes | |
| Visual no-code agent builder and AI copilot (AMP) | Not supported | Yes | |
| Built-in unified Memory system for agents | Not supported | Yes | |
| Production runs as sessions you can investigate and evaluate | Yes | Not supported | |
| First-class adapter (no CrewAI adapter yet: import a trace or build a project-local one) | Partial support | Not supported | |
| Replay with tool calls answered from the recording | Partial support | Not supported | Needs an adapter in the crew's process. An imported session replays passthrough only. |
| Cohorts: frozen, immutable session sets | Yes | Not supported | |
| Experiments: same cohort, one variable moved, compared | Yes | Not supported | |
| Deterministic and model-graded evaluators | Yes | Not supported | |
| Self-hosted eval server with no mandatory hosted control plane in the data path | Yes | Not supported | |
| Framework portability (record or import CrewAI, Pydantic AI, OpenAI Agents, LangGraph, or raw Python runs) | Yes | Not supported |
How the two surfaces map
| Concept | CrewAI | Kitaru |
|---|---|---|
| Layer | Agent harness (how the crew collaborates) | Replay-based evals (how you test what it did) |
| Core unit | Crew run (kickoff) | Session: recorded model and tool calls, re-executable |
| Composition | Standalone framework, AMP on top | Imported trace, or a project-local adapter |
| Tool calls in tests | Hit real systems, or you mock them by hand | Answered from the recording per tool policy, once an adapter is in the process |
| “Did my change help?” | Read the trace or AMP dashboard, compare by eye | Two runs over the same cohort, evaluated on both sides |
| Getting a session | No adapter yet: import the trace you already collect | First-class for Pydantic AI, OpenAI Agents SDK, LangGraph, Claude Agent SDK, Mastra, Vercel AI SDK |
| Where it runs | Your process, or AMP’s hosted control plane | Self-hosted server; workers replay in your environment |
Code comparison
# CrewAI has no first-class Kitaru adapter yet.
# Import the trace you already collect instead:
#
# kitaru agent register crew --command "python crew.py"
# kitaru session import crew-trace.jsonl \
# --importer kitaru/langfuse@latest \
# --agent crew@latest --tag imported-baseline --wait
#
# The imported sessions evaluate exactly like recorded ones.
# Freeze the ones that matter and run the evaluators:
#
# kitaru cohort create hard-crews --agent crew \
# --tag imported-baseline --display-version week-32
# kitaru session evaluate --cohort hard-crews@latest \
# --evaluator brief-quality@latest --evaluator kitaru/cost@latest --wait
#
# Replay with a model override needs an adapter in the
# crew's process; build one with kitaru-adapter-builder.from crewai import Agent, Task, Crew, Process
researcher = Agent(
role="Researcher",
goal="Produce a brief on {topic}",
backstory="Senior research analyst.",
llm="openai/gpt-4o-mini",
)
writer = Agent(
role="Writer",
goal="Write a draft from the brief",
backstory="Editor with a sharp ear.",
llm="openai/gpt-4o-mini",
)
research = Task(
description="Research: {topic}. Return a brief.",
expected_output="A short research brief.",
agent=researcher,
)
draft = Task(
description="Write a draft from the brief.",
expected_output="A short blog draft.",
agent=writer,
)
crew = Crew(
agents=[researcher, writer],
tasks=[research, draft],
process=Process.sequential,
)
# This kickoff produces a trace Kitaru can import.
result = crew.kickoff(inputs={"topic": "Q3 roadmap"})Put replay-based evals under
your CrewAI crew
If your problem is designing how a team of agents collaborates, CrewAI was built for it and worth adopting. If your problem is proving a change to the crew actually helped, Kitaru sits one layer underneath: import the traces you already collect, or build a small project-local adapter when you want native recording and replay.