8 Best Arize AI Alternatives for Agent Observability, Evaluation, and Replay

Arize AI is one of the most established names in ML and LLM observability, and on August 13, 2026, Dynatrace agreed to acquire it in a $915 million cash-and-stock deal. The deal has not closed as of September 2026. If your agents run on Arize today, that makes this a sensible moment to check what else the market offers.

The bigger reason to look is that observability is no longer one category. You might want tighter pre-release evals, open-source self-hosting for control over your data, or LangChain-native tracing if that is already your stack. Other teams treat simulation and human review as the core of their workflow. And a growing number want to replay what happened in production against the next version of their agent.

We reviewed the top 8 Arize AI alternatives with those different jobs in mind, so you can pick the one built for yours.

A Quick Overview of the Best Arize AI Alternatives

• Why look for alternatives: Some users report a steep setup and evaluation learning curve, while teams focused on simulation or eval-first workflows may prefer more specialized products. Arize’s evaluation index is built asynchronously and can lag by 1–2 hours, which can affect recently ingested spans used in evaluation runs.
• Who Should Care: AI engineers and platform teams running agents in production, especially anyone facing a model swap, a deprecation, or a prompt rewrite on a system they cannot safely experiment against.
• What to Expect: A breakdown of 8 Arize AI alternatives, from replay-based evals in Kitaru to eval-first platforms like Braintrust, open-source trace stores like Langfuse, and simulation tools like Maxim AI.
• Which Arize: Arize sells two products. Arize AX is the managed platform, with a free tier, Pro at $50 per month, and self-hosting only on Enterprise. Arize Phoenix is the self-hosted version under the Elastic License 2.0. It shares the same OpenInference instrumentation and is built as the on-ramp into AX. Each alternative below says which of the two it replaces.

The Need for an Arize AI Alternative?

Arize is a mature platform with a good enough reputation across regulated industries. The case for switching usually comes down to workflow fit.

1. Some Users Still Prefer Eval-First Workflows

A May 2026 PeerSpot review from a Technical Product Manager at HireRight described Arize as stronger on observability than experimentation, simulation, CI/CD gating, and benchmark management.

Treat that as a reviewer assessment rather than a current feature inventory: Arize AX now includes online and offline evaluations, datasets, experiments, trace and session evaluations, human annotation, trace replay, and full-agent experimentation.

Source

2. Phoenix Is the On-Ramp, Not a Standalone Open-Source Platform

If you run Phoenix rather than AX, the other complaints in this section mostly do not apply to you, but three others do. Phoenix is licensed under the Elastic License 2.0, which is source-available rather than OSI-approved open source, and it bars you from offering Phoenix as a hosted service. Monitoring dashboards, online evals, and the Alyx copilot are AX-only, so the moment you want production monitoring you are on the AX pricing page. Arize’s own docs describe the pairing as “start with Phoenix, grow into Arize AX”.

3. Traces and Eval Results Can Arrive After You Need Them

Arize’s own evaluator instructions say the evaluation index can lag by one to two hours. A little latency is no crime, but it’s harder to ignore when you changed a prompt five minutes ago, and now you have to wait hours to know whether it worked.

This is the kind of complaint that turns into a switch. In fact, price alone is less likely to push a small team away. Arize AX Free includes 25,000 spans per month and 15-day retention, so it’s a free entry point with decent usage. AX Pro is $50 per month for 50,000 spans and 30-day retention, the number to keep in mind when you read the prices below.

4. Setup and Instrumentation Takes Engineering Effort

Some users struggle to configure evaluations, dashboards, instruments, and almost the overall UI. At least, that’s what the reviews are saying. Arize AX is broad, and some reviewers still report a steep learning curve. However, instrumentation is not uniformly heavy: as of August 2026, AX natively supports OpenTelemetry GenAI semantic conventions, so applications already emitting compatible gen_ai.* spans can send them to Arize over OTLP without maintaining a custom attribute-mapping layer.

None of that is a design flaw. Arize AX is built for organizations running both classical ML monitoring and LLM observability across a bunch of teams. But for a five-person team without a dedicated platform or eval engineer, that breadth can feel heavier.

Source

Evaluation Criteria

We scored each alternative against three criteria:

Agent Observability and Tracing

We looked for tools that capture the complete agent traces across prompts, model calls, retrieval, tools, handoffs, and sub-agents. We also looked for support for sessions, multi-turn workflows, and streaming, so it’s useful for debugging real agent behavior.

Evaluation Depth and Reliability

We looked for offline and online evals, trace-level scoring, human review, code checks, and evaluator versioning. Bonus if the tool lets you check automated judges against human-reviewed examples.

Experimentation, Simulation, and Replay

We looked for tools that turn production traces into datasets, rerun failures, swap models, prompts, or tools, run multi-turn simulations, and compare quality, latency, and cost against a baseline.

What are the Top Alternatives to Arize AI

Arize AI Alternative	Best For	Key Features	Pricing
Kitaru by ZenML	Replay-based evals on production traces	- Replay real production sessions - Build immutable regression cohorts - Baseline-vs-candidate experiments	- Free (Open-source) - Paid plans start at $39/month
Braintrust	Eval-first teams that gate merges on scores	- Datasets, scorers, and experiments as the core objects - CI quality gates - Loop assistant for building eval cases from live traffic	- Free - Paid plans start at $249/month
LangSmith	LangGraph and LangChain teams	- Agent trajectory view and multi-turn threads - Datasets, LLM-as-judge, pairwise scoring, annotation queues - OTel ingest and export	- Free - Paid plans start at $39/seat/month
Langfuse	Teams that want the trace store on their own infrastructure	- MIT core, self-hosted free via Docker or Kubernetes - OTLP-native ingest - Prompt management, datasets, and experiments in the OSS core	- Free self-host, or Hobby cloud - Paid plans start at $29/month
Maxim AI	Testing agents before they reach production	- Multi-turn simulation across personas and scenarios - Evals at session, trace, and span level - Prompt CMS and a no-code evaluator library	- Free - Paid plans start at $29/seat/month
Galileo	Regulated teams that need to block bad output at runtime	- Luna distilled evaluator models - Real-time guardrails - Agentic metrics including tool-selection quality	- Free - Paid plans start at $100/month
Comet Opik	The cheapest hosted tier with a real OSS core	- Apache 2.0, same codebase self-hosted or cloud - 30+ built-in evaluation metrics - CI integration and agent trace trees	- Free (Open-source) - Paid plans start at $19/month
W&B Weave	Teams already standardized on Weights & Biases	- Scorers and LLM-as-judge harnesses - Evaluation playground - Agent trace tree with tool-call spans	- Free - Paid plans start at $60/month

1. Kitaru by ZenML

Kitaru by ZenML is an open-source replay-based evaluation runtime for AI agents. It takes complete production sessions and runs your agent’s real code again against controlled recorded tool history.

You can then test a changed model, prompt, model parameter, or code version on the same cases that produced the original behavior. And because Kitaru works as an underlying layer, your framework, trace store, and production runtime can stay where they are. Kitaru takes your recordings and re-executes your agent’s actual code against them, with exactly one variable changed. It replaces neither Arize product. AX or Phoenix stays as your trace store.

Feature 1. Replay Your Real Agent Against Recorded Production Behavior

When Kitaru replays a session, the run starts from the beginning with the original session inputs. And while model calls run again, tool calls can return recorded results so a refund or database write does not fire twice during a test.

The candidate can still choose a different path. Suppose it calls another tool or changes the arguments; the output is a result you can differentiate. You see what the agent would have done, next to what it actually did, on the same input.

Feature 2. Your Existing Trace Store Is the Input

Kitaru ships importers for Langfuse, LangSmith, Braintrust, Logfire, Arize Phoenix, and Kitaru JSONL, plus support for custom importers. The Phoenix importer reads JSONL exported from the Phoenix UI or JSON returned by the Phoenix CLI. So a team on Phoenix keeps Phoenix for tracing and adds replay on top.

The platform also wraps the harness you already chose. Recording adapters currently support PydanticAI, the OpenAI Agents SDK, and LangGraph in Python, plus the Vercel AI SDK and Mastra in TypeScript.

Feature 3. Build Regression Cohorts From Cases That Actually Happened

Group failed sessions, important production paths, and successful counterexamples into cohorts. Kitaru stores membership as immutable cohort versions, so every experiment runs against a fixed set of sessions that you can identify later. A production incident, for example, can become a permanent regression case instead of disappearing into an old trace.

Let’s say a support agent wrongly approves a $280 refund; you can keep that case beside valid refund sessions so the next prompt or model change has to fix the mistake without breaking the cases that already work.

Feature 4. Compare One Change Against the Baseline

A replay run holds everything constant except the variable under test. That means you can test one change across the same production-derived cohort.

Swap the model, edit the prompt, or register a new code version and see where behavior changed.

For example, test whether a cheaper model fixes the target failures without increasing tool errors or hurting response quality elsewhere. Wire that into CI and every prompt or model change gets tested against real production traffic before it ships.

Because baseline and candidate runs use the same production-derived cases and evaluator versions, you can see what improved, what regressed, and where the result is still inconclusive.

Pricing

Kitaru is free to self-host under Apache 2.0, with unlimited recording and imports, cohorts, evaluators, experiment runs, and replay on your own workers.

Its hosted Cloud plan includes a 14-day free trial, followed by:

• Cloud: $39 per month; Includes 3 agents, 2 seats, 90-day session retention, and every feature.
• Enterprise: Custom; Unlimited agents, custom seats and session retention, plus the governance layer of SSO, role-based access, and audit.

Pros and Cons

Kitaru’s strength is replay. Production runs can become regression cases, workers execute agent code in controlled environments, and your existing framework and trace store remain unmoved. It also costs almost nothing to try, because your current trace store is the input.

Kitaru is still beta (every giant was once a bloomer), and some Interfaces and adapter behavior can still change. It also does not replace Arize’s live monitoring dashboards and has little to replay before an agent reaches production.

2. Braintrust

Braintrust is the eval-first platform in this space, and somewhat ahead of Arize on experimentation. Where Arize starts with observability and adds evals, Braintrust puts datasets, scorers, and experiments closer to the center of development. It is an Arize AX replacement rather than a Phoenix one, since there is no open-source edition and self-hosting is Enterprise-only.

Features

• Build datasets from real test cases and production runs. You can mix hand-picked examples with cases pulled from live logs. Since datasets are versioned, you can always see exactly what each experiment tested.
• Score runs with custom code checks, LLM judges, or human review depending on what you’re measuring. The same scoring setup can work across offline tests and production logs, so you can keep quality checks under one evaluation model.
• See exactly what changed between experiments. You can compare prompts, models, or app changes side by side. Braintrust also surfaces outputs, scores, traces, and costs so you can spot which cases got better or worse.
• Use Loop, an assistant that builds eval cases from live traffic when you describe what you want in plain language. A product manager or support lead can then contribute cases without writing scorer code.

Pricing

Braintrust’s Starter plan has a $0 monthly platform fee. It includes 1 GB of processed data and 10,000 scores per month, with usage charges beyond those allowances. Apart from this it has 2 other plans:

• Pro: $249 per month
• Enterprise: Custom pricing (and the only tier with on-prem or self-hosted deployment)

Pros and Cons

Braintrust is the strongest for evaluation-first teams that want datasets, experiments, scorers, and CI checks to become one workflow. Strong model for turning production failures into future regression tests.

The trade-off is a premium entry point. There is no open-source edition; Pro starts at $249 per month, processed data and scores are metered separately, and self-hosting sits on Enterprise. If self-hosting is your concern, both products reserve it for higher tiers: Braintrust offers on-prem or hosted deployment on Enterprise, while Arize AX offers SaaS or self-hosted deployment on Enterprise.

3. LangSmith

LangSmith is framework-agnostic on paper but goes deepest if you build on LangGraph and LangChain. It combines tracing, datasets, offline and online evals, human review, and agent trajectory evaluation. Treat it as an AX replacement. There is no open-source edition, and self-hosting is gated to Enterprise, so it does not swap in for Phoenix.

Features

• LangSmith traces can capture nested model calls, retrieval steps, tool calls, and other child runs inside a trace. The trajectory view is built to view group-related runs as a persistent conversation, so you can inspect longer interactions and attach feedback at both run and thread level.
• Run offline evals on fixed datasets or score live production runs, or do both with code checks, LLM judges, or human feedback. Use this to catch regressions before shipping and spot issues that only appear with real users.
• Send tricky runs to human reviewers and collect structured feedback alongside automated scores with annotation queues. Use this for subjective checks like tone, usefulness, or whether an agent handled an edge case correctly.
• In case your team already sends telemetry to other observability tools, LangSmith can ingest and export OpenTelemetry, so traces stay portable even though the platform is proprietary.

Pricing

LangSmith’s developer plan is free for one seat and includes up to 5,000 traces. Other than that, it has two paid plans:

• Plus: $39 per seat per month
• Enterprise: Custom pricing

Pros and Cons

LangSmith fixes the two Arize complaints that hurt daily. Setup is fast, and traces show up the moment you run something, not ten minutes later. It’s solid for LangGraph and LangChain teams, with broad eval tooling plus LLM token usage and costs tracking for major providers.

LangSmith’s pricing can be overwhelming because seats, trace volume, retention, and platform units affect the bill. You are still trading one proprietary SaaS for another, with self-hosting gated to Enterprise. Watch the retention split as well, because base traces keep only 14 days and extended retention costs more per trace.

We covered the wider field in our guide to LangSmith alternatives.

4. Langfuse

Langfuse is one of the most widely adopted open-source AI engineering platforms, with roughly 33,000 GitHub stars as of September 2026. The core is MIT-licensed and self-hostable for free on Docker or Kubernetes, which makes it the default choice when open-source tracing and self-hosting are high on your list. That makes it the direct Phoenix swap, with an MIT core against Phoenix’s Elastic License 2.0, and its hosted Core tier comes in under Arize AX Pro.

Features

• Trace complete agent workflows with OpenTelemetry support. Capture model calls, tools, sessions, costs, latency, and other observations inside one trace model. Agent workflows can also be viewed as graphs, which makes longer tool-using runs easier to inspect than a flat list of calls.
• Manage prompts with versioning and deploy labels. You can link a prompt version back to the traces produced with it, test prompt or model variants in UI experiments, and compare their scores, cost, and latency before choosing one for production.
• Ingest traces through a native OTLP endpoint, so anything already emitting OpenTelemetry points at Langfuse with a config change rather than a re-instrumentation project.
• Judge a run using LLM-as-judge scoring, datasets, experiments, human annotation, and custom scores, that too in the open-source core.

Pricing

Langfuse is open-source and free to self-host under the MIT core license. It offers a free hosted Hobby plan with up to 50,000 units per month and 2 users. Plus, three paid cloud plans:

• Core: $29 per month
• Pro: $199 per month
• Enterprise: $2,499 per month

Pros and Cons

Langfuse is attractive when you want more control over data and cost. ClickHouse acquired it in January 2026, but open-source commitments are unchanged. That removes most of the usual acquisition risk.

That said, self-hosting still means running the underlying service. It leaves you to assemble pieces, so expect some real operational work at volume. Besides, the eval loop is lighter and less opinionated than Braintrust’s.

Our Langfuse alternatives guide goes deeper, and we have a direct Langfuse vs LangSmith comparison if you are choosing between the two.

5. Maxim AI

Maxim AI is a strong Arize AI alternative if you care about simulation, product collaboration, and evaluation workflows across the AI application lifecycle. It can help you generate multi-turn conversations across personas and scenarios and evaluate the agent against them before launch. It is an AX replacement. The evaluation platform is not open source, so Phoenix self-hosters get no equivalent here.

Features

• Simulate multi-turn conversations across defined personas and scenarios, then generate conversations across those conditions at scale. This helps when you are testing support, sales, or other agents where behavior develops slowly across several turns instead of one prompt-response pair.
• Build evaluators from a no-code library. QA and product people can then run eval cycles without an engineer in the loop. This is a direct answer to the Arize complaint about brittle custom-code evaluators.
• Test agents from different starting points. You can run simulations, dataset-based tests, or continue from part of a historical trajectory. This works well when you want to see how the agent handles a specific moment in a longer interaction.

Pricing

Maxim’s public pricing surface has shifted toward Bifrost, its gateway product, where the pricing page lists OSS and Enterprise.

Pros and Cons

Maxim is the strongest option here for pre-production testing, and the no-code evaluator library widens who can contribute to eval quality. Pricing is through and through free or Enterprise, and the tool-call chain evaluation is more agent-aware than most.

However, if your team wants an open-source, developer-first tracing tool, Maxim may feel broader than necessary. Larger teams will need to model the per-seat plus volume pricing. And simulated users are still simulated: they won’t reproduce the strange, expensive, real conversation that broke your agent last Tuesday.

6. Galileo

Galileo, now part of Cisco, is for teams that want agent evaluation to connect directly to production controls. It combines observability, evaluator models, agent-specific metrics, and runtime guardrails. It replaces AX rather than Phoenix, since it is proprietary and self-hosting is an Enterprise conversation.

Features

• Trace agent runs with Galileo Observe. It captures sessions, traces, and spans across LLM calls, tools, and other agent steps. You can follow the execution tree to see where a workflow slowed down, failed, or took an unexpected path.
• Block or modify unsafe output at runtime through configurable guardrails. You can take an action when a metric crosses the condition you set. That moves selected evaluations into the request path, which is useful when a violation needs to be stopped instead of reported later.
• Score with Luna distilled evaluator models. They run fast enough to sit in the request path, and they cost far less than judge calls to a large model. Or, create your own Custom Metrics. Galileo supports code-based metrics and custom LLM-as-a-judge metrics alongside.
• Score agent behavior with Galileo’s agent-specific metrics, including Tool Selection Quality, Tool Error, Action Advancement, Action Completion, Agent Efficiency, Agent Flow, Conversation Quality, Reasoning Coherence, and User Intent Change.

Pricing

Galileo offers a free forever cloud plan with 5,000 traces per month and unlimited users. On top, there are two paid plans:

• Pro: $100 per month
• Enterprise: Custom pricing

Pros and Cons

Galileo holds a strong enterprise posture with observability, experiments, datasets, guardrails, security, and flexible deployment options. The Luna evaluators are a real engineering advantage at volume, and the free entry point makes it an immediate option for migrating from Arize.

However, it’s not open-source, so anything beyond Free means a conversation about price. The fuller guardrail and deployment package sits on Enterprise, and it does not simulate multi-turn conversations. If your priority is self-hosting or replaying historical sessions against new agent code, Kitaru will fit more closely.

7. Comet Opik

Opik is Comet’s open-source LLM observability and evaluation project. It is Apache 2.0 licensed, and with more than 21,000 GitHub stars, it is the second-largest dedicated LLM observability project, behind Langfuse. It is the other direct Phoenix swap, with Apache 2.0 rather than Elastic License 2.0, the same codebase self-hosted or in the cloud, and the cheapest hosted tier in this list.

Features

• Debug multi-step agent runs through trace trees, with framework-agnostic instrumentation. Opik records a span tree for each traced run, including LLM calls, tool invocations, outputs, feedback, and latency. You can start from a bad production response and work backward through the steps that produced it.
• Measure quality with datasets and more than 40 prebuilt metrics. This includes quantitative experiments across LLM, RAG, and agent use cases, plus custom metrics and a HITL annotation queue when the defaults do not fit.
• Create Test Suites combining natural-language assertions with execution policies for pass-or-fail testing. An LLM judge checks those assertions, while execution policies control repeated runs and how many must pass.

Pricing

Comet provides an open-source version with unlimited spans, retention, and seats, and a free forever cloud plan with up to 25,000 spans per month and 10 team members. Then, you can scale via two paid plans:

• Pro Cloud: $19 per month
• Enterprise: Custom pricing

Pros and Cons

Opik is one of the lowest-cost hosted options here, and its Apache 2.0-licensed open-source platform gives teams a permissive self-hosting path. At $19 per month for 100,000 spans, it is an order of magnitude below Braintrust Pro, and the free self-hosted path has no usage ceiling at all. Its Test Suites also provide a useful bridge between trace debugging and regression checks.

Opik now goes beyond scoring stored outputs. Test Suites can turn production failures into regression cases and run them against an updated task or agent, while the Agent Playground can execute connected agents from the UI. What it does not provide is Kitaru-style full-session replay against recorded external tool history.

Our Comet alternatives piece covers the wider Comet platform.

8. W&B Weave

If you’re using Weights & Biases, Weave brings tracing, datasets, evaluations, scorers, feedback, and experiment comparisons into the same vendor stack. Weave takes the experiment-tracking rigor Weights & Biases built for classical ML and applies it to LLM evaluation. It is an AX replacement for teams already on Weights & Biases. Self-hosting sits at enterprise licensing, so it is not a Phoenix swap.

Features

• Records application calls with inputs, outputs, timing, errors, and parent-child relationships. That gives you a trace view for debugging multi-step applications without moving the work outside the W&B environment.
• Compare models and scorers without writing any evaluation script in the Evaluation Playground. You can load a dataset, add several models, attach LLM judges, and run them side by side in the UI.
• Define scorers and LLM-as-judge metrics with the same versioning discipline W&B applies to experiments. Weave supports function- and class-based scorers, and also ships predefined scorers and local small-model scorers for checks such as hallucination, toxicity, coherence, and context relevance.

Pricing

W&B pricing includes Weave capabilities across W&B plans, starting with a free hosted plan, plus two paid plans:

• Pro: $60 per month
• Enterprise: Custom pricing

Pros and Cons

Weave versions, scores, and compares evaluations the same way W&B does ML experiments. If W&B is already your system of record for models, the continuity is worth something.

The ingestion meter is the problem for agent workloads. At $0.10 per MB, overage works out to roughly $100 per GB. Multi-turn agents that resend accumulated context on every turn are precisely the traffic shape that meter punishes. Self-hosting sits at enterprise licensing. Standalone Weave adoption is also small compared to the parent brand. We go into more detail in our Weights & Biases alternatives guide.

The Best Arize AI Alternatives for Agent Teams

There is no single Arize AI replacement because these products solve different parts of the problem.

• If you run Phoenix and want an OSI-approved license, or a hosted tier that is not AX, Langfuse and Opik are the direct swaps.
• If you run AX and evaluations, datasets, and CI checks drive development, choose Braintrust, or LangSmith when LangGraph already sits at the center of your stack.
• If you run AX and the gap is before production or at runtime, Maxim covers simulation and Galileo covers guardrails.

And there is a fourth problem underneath all of those. You can see everything your agent did and still not know whether the change you are about to ship is safe. That is a different job, and that is what Kitaru solves. Point it at the traces you already have in Langfuse, LangSmith, Braintrust, or Logfire, replay last month’s real sessions against the model swap you have been putting off, and see what breaks before your users do.

Book a demo with our founding engineer to see Kitaru in action!

Or try it yourself! It’s open source: zenml.io/product/kitaru.