Deutsche Telekom successfully deployed LMOS (Language Models Operating System), one of Europe's first enterprise agentic platforms in 2023, by prioritizing existing teams and technology stacks over trendy frameworks. The company built a JVM-based agentic framework using Kotlin that integrated with existing APIs, observability tools, and DevOps practices, while introducing an Agent Definition Language (ADL) to enable business users to define requirements directly. The platform went live across multiple countries, demonstrating that successful enterprise AI deployments require compressing fault lines between teams, platformizing hard infrastructure concerns, and enabling existing engineers rather than creating isolated AI teams with novel tech stacks.
This case study presents the journey of building LMOS (Language Models Operating System) at Deutsche Telekom, one of the first enterprise agentic platforms deployed in Europe in 2023. Arun Joseph, who led the AI engineering program at Deutsche Telekom before founding Masaic, shares detailed insights into what actually works when deploying LLMs in production within large enterprises. The presentation distinguishes between two classes of agentic systems: Level 1 agents focused on chatbots and automation, and Level 2 operational intelligence systems that perform complex decisioning and actioning across enterprise workflows.
The core thesis challenges the common approach of building AI systems with specialized teams using cutting-edge frameworks, instead advocating for leveraging existing engineering teams, technology stacks, and organizational knowledge. Deutsche Telekom’s success stemmed from recognizing that enterprise AI is fundamentally about crossing organizational fault lines—the boundaries between teams, systems, and domains—rather than simply implementing the latest research papers.
Joseph frames the enterprise AI problem through the concept of “fault lines,” which represent boundaries between teams, systems, and domains. In typical enterprise architecture, any work can be summarized into four packages: storage layer, transformation layer, presentation layer, and transmission (events or APIs). Each API boundary represents a potential fault line where different teams own different services, each with their own expertise, processes, and technologies.
The fundamental issue with many enterprise AI initiatives is that they create new teams detached from domain expertise. A specialized AI team building a product ordering agent, for example, faces massive challenges when the underlying APIs follow complex specifications like TM Forum standards in telecommunications. These APIs are messy, poorly documented, and understood by only a few domain experts within specific fault lines. Creating a separate AI team with the latest frameworks but no domain knowledge leads to endless waiting for API access, documentation, and domain expertise—effectively guaranteeing failure.
Joseph illustrates this with a vivid metaphor: enterprises aren’t building new aircraft from scratch where you’d bring in aeronautical engineers. Instead, they need to take existing cars and trucks (legacy systems) and make them fly. This requires working with the people who built and maintain those systems, not replacing them with external specialists.
A second major challenge is the explosion of AI tooling that creates operational nightmares. Joseph provides a satirical but realistic example where a single line of code using decorators for evaluation, telemetry, and memory tools translates into five separate containers. In one consulting engagement, he witnessed a procurement decision that resulted in 25 containers just to support an evaluation tool, leaving the DevOps team completely overwhelmed.
This “vendor containers for line of code” problem creates new fault lines between AI developers and operations teams. When each AI capability—evaluation, telemetry, memory—comes from a different vendor with their own SDK and infrastructure requirements, the existing DevOps teams cannot support or understand the deployment requirements. The licensing costs are often trivial compared to the operational complexity introduced.
Deutsche Telekom’s approach started with a small team of five distributed systems engineers in 2023, before “agents” became a widespread term. The critical decision was choosing a technology stack that worked with existing engineers rather than forcing a paradigm shift.
The entire Deutsche Telekom API ecosystem was built on Java/JVM, with hundreds of client libraries for invoking these APIs and comprehensive observability infrastructure supporting the JVM ecosystem. The team initially explored LangChain but found it unsuitable for their needs. Instead, they built their own framework called ARC (Agent ReaCtor) using Kotlin, specifically designed to work within the existing JVM ecosystem.
This decision allowed product API team members to immediately test and contribute to agent development using familiar tools and libraries. By attaching existing enterprise libraries as dependencies, the learning curve was dramatically reduced. Engineers who knew how to invoke the product API could now do so within the agent framework without learning entirely new paradigms.
For the DevOps and operations teams, LMOS integrated directly with existing observability infrastructure—Grafana, Prometheus, SigNoz, and distributed tracing systems—rather than requiring specialized AI observability tools. A single LLM API call generates extensive telemetry (tokens, tool calls, traces), and requiring specialized systems to understand this creates an unsustainable operational burden.
By plugging telemetry into existing systems using standard protocols like OpenTelemetry, the operations team could manage the AI infrastructure with their existing skillsets and tools. This “please don’t come to us with more containers” approach was critical for operations team buy-in and sustainable deployment.
The overall architecture consisted of:
The framework abstracted the hard parts of agent development while exposing familiar programming patterns to JVM developers. This allowed rapid iteration without requiring engineers to become experts in the latest AI frameworks.
A critical innovation was recognizing that traditional requirements processes don’t work for agentic systems. Previously, business users would write Jira tickets requesting new buttons or features, which developers would implement. But how do you write requirements for a chatbot that handles fluid, natural language inputs?
Deutsche Telekom developed an Agent Definition Language that enabled business users to define agent behaviors in a structured but accessible format. The ADL environment runs as a simple Spring Boot container with a UI, allowing business users to:
For example, in a Volkswagen service bot scenario, business users could define the use case for “car broke down, need appointment” including the expected flow, alternate paths if appointment dates aren’t available, and fallback behaviors. The use case structure included three key constructs: the solution, the alternate solution, and fallback logic.
Critically, ADL definitions aren’t sent directly to the LLM as raw prompts. Instead, an ADL compiler processes these definitions, performing optimizations similar to tree-shaking in JavaScript bundlers. The compiler generates optimized system prompts, compressing and structuring the business logic for efficient LLM consumption.
This approach compressed the loop between business requirements and implementation. Business users could iterate on agent behaviors without creating Jira tickets or waiting for development cycles. Engineers focused on wiring APIs and infrastructure, while business users defined the domain logic and customer experience.
LMOS went live in 2023 across multiple countries within Deutsche Telekom. While specific numbers around agent handover rates aren’t public, the platform demonstrated:
The platform has since moved to the Eclipse Foundation as an open-source project, continuing to run in production at Deutsche Telekom.
After LMOS, Joseph’s new company Masaic focuses on more sophisticated “Level 2” agentic systems for operational intelligence. These systems go beyond chatbots and simple automations to provide decisioning and actioning capabilities across complex enterprise workflows.
The case study presents a detailed example from heavy machinery operations (companies like Hitachi, John Deere, SENNEBOGEN). The goal is reducing machine downtime by ingesting diverse data sources:
A service operations manager can query the system: “We’ve seen a spike in error code X in the Midwest, what can we do about it?” The system then:
This represents a decisioning system that combines analysis, recommendation, and action orchestration—far beyond simple question answering.
The operational intelligence system uses ephemeral agents rather than pre-defined agent types. When a query comes in, the system:
This approach scales better than creating dozens of named agents (fault analysis agent, testing agent, root cause analysis agent, etc.). The ephemeral model allows the system to dynamically construct the agent architecture needed for each query.
As the team built 50-60 agents for heavy machinery operations, they recognized that most agent code was plumbing rather than business logic. Developers spent 95% of time on infrastructure concerns (observability, guardrails, evaluations, model integration) and only 5% on actual agent logic.
The team systematically moved hard parts into a platform layer, including:
This platformization dramatically reduced the size and complexity of individual agents, allowing developers to focus on business logic.
After building 100+ agents, the team identified a core programming paradigm: an agent is simply a loop that takes a goal, operates with context, accepts constraints, and executes tools (which may construct additional tools). This loop continues until the goal is achieved or human-in-the-loop intervention is required.
By moving this agent loop primitive into the platform and exposing it as a single API (similar to how S3 exposes object storage or Stripe exposes payments), agents became ephemeral constructs defined by goals rather than pre-built components.
The evolved platform, called AgC (Agentic Compute), exposes compute itself as an agent-making machine. Key characteristics:
The presentation includes several live demonstrations:
Demo 1: Business Proposal Generation An agent helps a salesperson draft a B2B proposal after a call with a company. The system:
Importantly, no LangChain, LangGraph, or CrewAI code was written. The developer only attached the necessary tools and provided instructions.
Demo 2: Dynamic Tool Integration The presenter demonstrates adding a completely unrelated MCP tool (Shopify’s Allbirds shoe catalog) to the same agent with zero code changes. After adding the MCP configuration, the agent can immediately use the new capability to add shoes to business proposals—demonstrating true compute-as-agent flexibility.
Demo 3: Model Switching The platform allows switching LLM providers/models with a single configuration change, demonstrating the abstraction of model choice from agent logic.
Demo 4: Telemetry Fan-Out All agent interactions automatically generate telemetry distributed to configured backends (Langfuse, observability platforms, etc.) through OpenTelemetry collectors, without requiring application-level instrumentation.
Demo 5: Remote Tool Execution The system supports attaching tools running on local machines (demonstrated with Salesforce updates requiring local browser automation for 2FA) to agents running in the platform, showing flexible tool execution models beyond simple MCP servers.
Joseph strongly advocates starting with Level 1 agents (fluid input orchestration) rather than attempting complex autonomous systems immediately. Level 1 agents provide significant value by:
In large enterprises, “bot farms” built with workflow orchestration engines (Camunda, etc.) require dedicated engineers for each new automation. Level 1 agents democratize this capability, allowing business users to create automations without workflow engineering expertise.
The core lesson from LMOS is that successful enterprise AI requires working with existing teams, stacks, and organizational structures rather than creating isolated centers of excellence. Engineering expertise in distributed systems, APIs, and operations is not going away—it needs to be integrated into AI systems, not replaced.
ADL represents a paradigm shift: business users don’t need more user interfaces, they need a language for expressing requirements that directly translates into executable programs. This compression of the requirements-to-implementation loop is critical for sustainable agent development.
The evolution from individual agents to platform primitives demonstrates that successful LLMOps requires treating infrastructure concerns (observability, model management, tool execution, security) as platform responsibilities rather than application concerns.
The ultimate abstraction exposes the compute layer itself as an agent-making capability, allowing ephemeral agent construction based on goals rather than pre-defined agent architectures. This scales far better than maintaining libraries of named agents.
While the presentation advocates strongly for certain approaches, several trade-offs are implicit:
Technology Lock-in vs. Familiarity: Choosing JVM/Kotlin locked Deutsche Telekom into that ecosystem, but enabled rapid adoption. Organizations with different primary stacks would need different frameworks.
Custom Framework vs. Ecosystem Tools: Building ARC and LMOS meant maintaining custom code rather than leveraging ecosystem tools like LangChain. This trades maintenance burden for better enterprise fit.
Platform Complexity: While individual agents became simpler, the platform layer grew more complex. This requires skilled platform engineers and creates a different scaling challenge.
Business User Programming: ADL requires business users to think programmatically about agent behavior. Not all business users have this skillset or inclination.
Open Source Strategy: Moving LMOS to Eclipse Foundation demonstrates commitment to open source, but also means reduced competitive differentiation for Deutsche Telekom.
The case study represents a pragmatic, production-focused approach to enterprise LLMOps that prioritizes organizational fit, operational sustainability, and incremental value over theoretical purity or cutting-edge techniques. The success of LMOS demonstrates that enterprise AI adoption is fundamentally an organizational and architectural challenge rather than purely a technical one.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Notion, a knowledge work platform serving enterprise customers, spent multiple years (2022-2026) iterating through four to five complete rebuilds of their agent infrastructure before shipping Custom Agents to production. The core problem was enabling users to automate complex workflows across their workspaces while maintaining enterprise-grade reliability, security, and cost efficiency. Their solution involved building a sophisticated agent harness with progressive tool disclosure, SQL-like database abstractions, markdown-based interfaces optimized for LLM consumption, and a comprehensive evaluation framework. The result was a production system handling over 100 tools, serving majority-agent traffic for search, and enabling workflows like automated bug triaging, email processing, and meeting notes capture that fundamentally changed how their company and customers operate.
Stripe, processing approximately 1.3% of global GDP, has evolved from traditional ML-based fraud detection to deploying transformer-based foundation models for payments that process every transaction in under 100ms. The company built a domain-specific foundation model treating charges as tokens and behavior sequences as context windows, ingesting tens of billions of transactions to power fraud detection, improving card-testing detection from 59% to 97% accuracy for large merchants. Stripe also launched the Agentic Commerce Protocol (ACP) jointly with OpenAI to standardize how agents discover and purchase from merchant catalogs, complemented by internal AI adoption reaching 8,500 employees daily using LLM tools, with 65-70% of engineers using AI coding assistants and achieving significant productivity gains like reducing payment method integrations from 2 months to 2 weeks.
