Evals your risk team
can stand behind

Our AI engineer reads your agent's traces next to your evals and your procedures, and shows you what the evals miss.

Request demo
An example of what your AI engineer tells you about a KYC review agent, Oct 21 to 27. The team's evals pass 93% of last week's cases; by the team's own procedures, 71% should pass. A completeness judge passed 33 cases with no source-of-funds check, 46 cases treated adverse-media hits older than 12 months as clear, and 21 closed a PEP match without a second reviewer. No eval checks the last two.

The problem

The dashboard says 93%.Analysts still check every case.

Your agent reviews KYC cases, drafts credit memos or answers clients, and its traces already land in your tracing tool. Nobody can say which of its evals to trust.

  1. 01

    Procedures move, judges don't

    Policies change with every audit finding. The judges were written against the old ones.

  2. 02

    Second line asks for proof

    Risk and compliance want to see how a change was tested before it ships. A pass rate is not an answer.

  3. 03

    Nobody wants the job

    Every hour an analyst spends grading an agent is an hour not spent on cases.

How it works

You answer a few questions. Good evals fall out.

You don't write evals from scratch. Your AI engineer asks only what it can't work out alone.

  1. 01

    Connect your repos

    Install the GitHub app and add your tracing keys. It finds your agent, the evals you already run and the procedures they should follow.

  2. 02

    A short interview

    Your AI engineer asks you a few questions about what it found, in the browser or in Slack. Tap to answer.

  3. 03

    Hear what your evals miss

    Right after, it tells you what your evals missed last week, in the order you said you care about. More follows in Slack.

  4. 04

    Fifteen minutes a week

    Three real cases and about ten quick questions. Your answers become a calibrated check and a frozen set of cases.

Security

Built for regulated data

Run it on-prem and customer records, transactions and case files never leave your walls. You choose where it runs and which models it calls.

SOC 2 Type II BadgeISO/IEC 27001:2022 certification badge
Visit the trust center
  1. 01

    Runs where you need it

    Your AI engineer runs in our cloud by default, or on-prem in your own cloud, air-gapped if you need it. Kitaru is open source and runs on your machine.

  2. 02

    Your models, your keys

    Judges and replays call the providers you choose with your own keys, including models you host yourself.

  3. 03

    SOC 2 Type II and ISO 27001

    Independently audited for security, availability and confidentiality.

  4. 04

    SSO, roles and audit logs

    SAML or OIDC single sign-on, SCIM provisioning, custom roles and org-wide audit logs.

  5. 05

    Your encryption keys

    Bring your own keys for encryption, and choose the region your data lives in.

  6. 06

    A record of every change

    Reviewed sessions, verdicts and replays are kept, so you can show how each change was tested before it shipped.

Trusted to run production AI, from pipelines to agents, at

  • AXA
  • JetBrains
  • ADEO
  • Leroy Merlin
  • Brevo
  • Safran
  • AECOM
  • Airbus Defence & Space
  • Rohlik
  • Knuspr
  • Maven Robotics
  • CrossScreen Media
  • GEMA
  • Homa Games
  • Koble
  • IKEA
  • Sciemo
  • Vodafone
  • Stepstone
  • Neara
  • Rivian
  • Happening XYZ
  • Veridas

Questions from security and risk

Where do our traces go?
Your traces stay in the tracing tool you already use. By default, your AI engineer runs in our cloud and reads them through keys you can revoke. Deploy it on-prem and it runs inside your own cloud, so nothing leaves your infrastructure.
Can we use our own models?
Yes. Judges and replays call the providers you pick with your keys, and you can point them at models you host yourself.
Which certifications do you hold?
ZenML holds a SOC 2 Type II report and ISO 27001 certification. Reports and policies are in our trust center.
Can it run air-gapped?
Yes. ZenML Pro supports multi-region and air-gapped deployments. See deployment options.
Can we show model risk how it was tested?
Yes. Every reviewed session, verdict, check and replay is kept with who decided and when. Export it as evidence for model validation, your model risk committee or an audit.
Which agents does it work with?
Any agent whose traces you already collect: KYC and onboarding review, credit memos, fraud triage and client service. It reads traces from Langfuse, LangSmith, Braintrust, Logfire, MLflow or Phoenix.
Where is our data hosted?
On hosted plans you choose an EU or US region, and we process data under GDPR with a data processing agreement. On-prem, everything stays in your own cloud.
What about DORA?
Your AI engineer keeps the testing records and audit trail that DORA and model risk reviews ask for, and on-prem it runs inside your own ICT estate. It doesn't make you compliant on its own; it gives your second line the evidence.

Bring one agent
and a week of traces

We'll show you what your evals miss on your own data, and which checks your risk team would actually stand behind.

Request demo