Uber built a comprehensive managed software factory powered by agentic AI to accelerate software development across thousands of engineers in 12 global tech sites. The solution consists of six core building blocks: a model gateway for secure API access with PII redaction, an MCP gateway for unified tool access, agentified cloud development environments, a managed skills marketplace, a context graph connecting 40 million entries across Uber's infrastructure, and an AI assistant called Cortana. This infrastructure enabled over 70% of pull requests to be generated by AI agents, doubled lines of code per engineer year-over-year, and automated 250 migrations totaling 9 million lines of code. The system supports the entire software development lifecycle from ideation through maintenance, with capabilities for autonomous coding, self-healing CI/CD, and automated code review.
Uber has developed an extensive managed software factory powered by agentic AI that represents one of the most comprehensive production deployments of LLM-based development tools at enterprise scale. The initiative spans thousands of engineers across 12 global technology sites and has fundamentally transformed how software is developed at the company. The reported impacts are substantial: over 70% of pull requests are now generated by either local or cloud-based AI agents, engineers are producing twice the number of lines of code year-over-year, and the system has automated over 250 migrations involving 9 million lines of code. While these metrics are impressive, it’s important to note that quantity metrics like lines of code and PR volume don’t necessarily translate to business value or code quality, though Uber’s emphasis on validation and quality gates suggests they’re aware of this concern.
The foundation for this transformation was laid over six years through investments in monorepo architecture and the Bazel build system, which provided the structural consistency needed for automated tooling to work effectively. This highlights an important LLMOps lesson: the success of AI-powered development tools often depends heavily on existing infrastructure investments that enable standardization and automation.
The model gateway serves as the central infrastructure layer for all LLM interactions at Uber, handling over 100 million model requests per day from 800+ internal projects. This component addresses three critical production requirements: data privacy, latency constraints, and cost attribution.
The gateway provides a single OpenAI and Anthropic-compatible endpoint that all internal and external use cases route through, ensuring consistent policy enforcement. The middleware stack includes several layers: identity and authentication using SPIFFE/SPIRE, a data anonymizer that redacts over 20 PII types, and an AI guard component consisting of five specialized models that enforce safety and policy requirements. Critically, the entire middleware stack operates under 100 milliseconds of latency, which is essential for maintaining acceptable user experience in interactive coding scenarios.
From an LLMOps perspective, the gateway’s attribution capabilities are particularly noteworthy. Every request is associated with a specific project in Uber’s internal catalog and can be attributed to individual users, teams, and callers in real-time and in the data lake. This enables sophisticated cost management, including spend views and guardrails across the entire LLM portfolio. The system also captures audit logs and session traces, which feed into benchmarking and self-improvement loops.
The gateway supports both frontier models from major vendors and open-source models, whether hosted on Uber’s infrastructure or by external providers. The architecture also implements various caching and token optimization strategies at this layer, though specific details of these optimizations weren’t provided. For developers, the interface is straightforward: use a standard client, set a project ID, and the gateway handles everything else. This abstraction is crucial for scaling AI adoption across thousands of engineers without requiring deep expertise in prompt engineering or LLM operations from every developer.
The Model Context Protocol gateway addresses the challenge of making Uber’s thousands of internal APIs and numerous SaaS tools accessible to AI agents. Initially, none of Uber’s internal APIs were agent-accessible, and each SaaS tool had different authentication and setup requirements, creating significant friction for agent development.
The MCP gateway implements middleware similar to the model gateway pattern, providing a centralized point for tool access. An automated crawler examines internal APIs and projects them into MCP format with a single configuration change, dramatically reducing the effort to make new tools available to agents. The gateway also handles SaaS integrations with services like Google, Slack, and Jira, managing hosting and token exchange so engineers have one common installation method for all MCPs.
A particularly interesting aspect of the MCP gateway is the evolution of token optimization strategies. Initially, the system used direct MCP patterns, but token consumption was problematic. Uber developed “Omni MCP,” a single MCP that can discover and invoke any other MCP within the gateway, reducing the context overhead. Subsequently, they introduced a CLI pattern projection where even the response doesn’t consume context window space. Most recently, they added a code mode skill that automatically generates Python scripts on-the-fly to hyper-optimize high-token-consumption use cases.
These optimizations resulted in 40% fleet-wide token savings, demonstrating that even with the declining cost of tokens, optimization remains important at scale. The system now provides access to over 1,000 MCP tools. From an LLMOps perspective, this illustrates the importance of treating the tool layer as infrastructure requiring the same level of investment as the model layer itself, rather than as an afterthought.
Uber’s devpod infrastructure predates their AI initiatives, originally created to handle large monorepos containing millions of lines of code. The company has now “agentified” these environments to support autonomous agent execution. The key requirements are speed, isolation, global availability across sites, and the ability to spawn many instances.
The solution uses pre-provisioned Kubernetes balloon pods that are ready when agents need them. These environments come with repositories already snapshotted and search indexes pre-built, allowing agents to start working within seconds rather than minutes. This pre-provisioning strategy is crucial for maintaining the responsiveness needed in agentic workflows where delays compound as agents iterate.
An important architectural evolution was the move from language-specific devpods (for Go, Java, Android, etc.) to “mega devpods” that contain all repositories in a common environment. This change reflects a fundamental shift in how development happens: roles are blurring as agents work across repositories and engineers increasingly need cross-repository visibility. The mega devpod approach also supports non-engineer employees who may need to work with agent harnesses but shouldn’t have to understand the intricacies of different language toolchains.
From an LLMOps perspective, the devpod architecture demonstrates the importance of execution environments in production AI systems. Agents need more than just API access and context—they need actual runtime environments where they can build, test, and validate changes. The seconds-to-ready requirement shows that even small latencies can significantly impact the viability of agentic workflows.
The skills marketplace addresses problems that emerged as engineers began building numerous agent skills across many repositories: duplication of effort, poor discoverability, inconsistent configuration, and variable quality. The managed marketplace provides lifecycle management for both core skills and domain-specific skills, currently hosting 2,500 skills that collectively see over 20,000 executions per day.
Quality control is implemented through automated lint checks and reviews that ensure baseline skill quality before skills are accepted into the marketplace. Discovery and installation are simplified to a single command, and skills are auto-installed based on engineer personas, allowing agents to automatically access appropriate skills without manual intervention.
The marketplace also implements continuous evaluation by collecting traces and comments, feeding this information back to skill authors for improvement. This feedback loop is critical for maintaining and improving skill quality over time, though the presentation notes this is an area of active investment, suggesting the continuous improvement mechanisms are still maturing.
From an LLMOps perspective, the skills marketplace represents an important pattern for scaling agentic AI: treating skills as managed artifacts with proper governance, quality gates, and feedback loops, rather than as ad-hoc scripts scattered across repositories. The challenge of skill quality and the need for automated quality checks highlights that even with AI assistance, human oversight and structured processes remain essential. The scale—2,500 skills and 20,000 daily executions—suggests this is a substantial investment that goes well beyond a prototype or proof of concept.
The context graph addresses a fundamental challenge observed in agent execution traces: agents spent excessive time gathering basic context in large monorepos, trying to determine where services are located, what their dependencies are, who owns them, and what patterns to follow. This information was scattered across 20-30 different systems at Uber, each requiring separate skills and MCPs to query, resulting in high token consumption, increased latency, and unpredictable outcomes.
The context graph consolidates all information about how Uber runs into a single unified graph structure containing 150 unique node and edge types with 40 million entries. The graph spans from mobile app construction to backend services to the data lake, and includes design documents, Jira tickets, incidents, bugs, and other artifacts. This comprehensive approach enables agents to quickly locate relevant context without extensive exploration.
The graph is being integrated into various use cases including on-call root cause analysis, planning, data analysis, and security scans. An example query—“how many mobility trips in India are cash”—demonstrates the value: with the graph, the system can understand the concepts involved (mobility, geography, payment method), identify relevant tables, and construct appropriate SQL. Uber reports massive improvements in tokens, turns, and latency across evaluations when using the graph compared to without it.
From an LLMOps perspective, the context graph represents a significant architectural pattern for production LLM systems: rather than relying solely on retrieval-augmented generation with vector embeddings, Uber has invested in a structured knowledge representation that captures relationships and semantics explicitly. This approach likely provides more reliable and predictable results than pure embedding-based retrieval, particularly for complex queries that require understanding relationships across multiple systems. However, building and maintaining a 40-million-entry graph with 150 node and edge types represents substantial engineering investment, and the ongoing maintenance burden—keeping the graph synchronized with actual infrastructure changes—is likely considerable.
Cortana serves as the unified interface layer that packages all the previous building blocks for company-wide use. It integrates skills, MCPs, and the context graph, making them accessible through multiple surfaces: Slack, CLI, and web interfaces. This multi-surface approach ensures that AI assistance is available wherever employees work, rather than requiring them to switch to a specialized tool.
The system allows employees to ask questions that can span looking up the context graph, invoking skills, and examining code across any codebase. An important capability is personalization: employees can configure Cortana with custom skills and prompts and integrate it into team Slack channels, where it can act as a knowledgeable team member with context about that specific team’s work. In the first month of this capability, 300 unique personas were created, and the system handled over 20,000 sessions per day, suggesting strong adoption.
The multi-modal interface is particularly important from an LLMOps perspective. Slack integration enables collaborative work where multiple people can interact with the same session, which is natural for team-based software development. The CLI interface likely serves developers who prefer terminal-based workflows, while the web interface provides a richer environment for complex interactions. This recognition that different users and use cases require different interfaces is a mature approach to deployment that goes beyond simply providing an API or a single chat interface.
The presentation demonstrates how these building blocks come together through an example feature: creating better pickup locations for riders leaving large venues like World Cup stadiums. The workflow begins with ideation in Slack, where Cortana is tagged to help evaluate whether this is a good business opportunity. Using the context graph, Cortana can analyze historical data about large-scale venue events and identify relevant stadiums, compressing what could have taken weeks of research and alignment into a much shorter timeframe.
The workflow proceeds through product requirements, where Cortana helps determine appropriate rollout scope, and then to Figma design mock-ups, creating two variants for A/B testing. Cortana then assists with design decisions about what existing code can be leveraged in both frontend and backend. The presentation suggests this prototype phase happens very quickly compared to traditional approaches.
Implementation is handled by Minion, Uber’s cloud coding agent, which can operate in both interactive and autonomous modes. Minion works within the agentified devpod infrastructure, enabling it to make changes across repositories, handling both backend and frontend modifications. Critically, Minion stops at creating a draft PR without pushing to CI, which represents an important design decision based on experience: while autonomous coding works well for toil reduction, building advanced end-to-end features requires validation before consuming CI resources.
Uber has implemented a sophisticated validation strategy that distinguishes between inner loop and outer loop checks in the software development lifecycle. The inner loop refers to pre-CI validation, while the outer loop includes CI and code review. A key insight driving their architecture is “shifting left”—moving more checks into the inner loop to catch issues earlier and reduce load on CI infrastructure.
Inner loop checks now include traditional static analysis with automatic fixes when issues are detected, but also more sophisticated validations. Visual validation launches a simulator, captures screenshots, and compares them to Figma specifications. Integration validation brings up the backend service in a staging environment and validates frontend-backend integration. Even code review has been partially shifted left, with a smaller, faster model performing initial reviews in the inner loop.
The outer loop still includes CI, but with self-healing capabilities where the system automatically fixes many errors that occur during continuous integration. Outer loop code review uses more powerful models with reasoning capabilities and skills for deeper analysis. For autonomous diffs from Minion, the PR includes a table showing all validation checks performed, including screenshots, giving human reviewers confidence that the code has gone through extensive self-improvement beyond the initial generation.
This validation architecture addresses a critical LLMOps challenge: autonomous code generation is only valuable if the generated code is correct and maintainable. The multi-layer validation approach with both automated checks and human oversight represents a mature production pattern. However, it’s worth noting that the effectiveness of these validations depends heavily on the quality of the checks themselves, and maintaining Figma-to-screenshot comparisons, comprehensive integration tests, and effective self-healing CI rules represents ongoing engineering investment.
Beyond initial feature development, Uber has implemented managed maintenance loops for ongoing code health. Services and features can be enrolled in maintenance skills that run on scheduled intervals. An example is feature flag cleanup: after A/B testing determines that one variant is preferred, a skill can automatically remove the other variant’s code. These loops are managed through a centralized surface rather than allowing thousands of ad-hoc loops across the company, which prevents resource overload and coordination problems.
Maintenance skills run on Sundays when CI capacity is more available, and the number of diffs produced is controlled to avoid overwhelming engineers on Monday morning. The diffs generated by maintenance skills receive comments and are either landed or rejected, creating labeled data that feeds back into skill improvement—another example of the continuous evaluation loops Uber is building throughout their system.
At a monthly cadence, Uber reviews incidents to identify patterns that could be addressed by new maintenance skills, creating a virtuous cycle where operational problems lead to automation that prevents future occurrences. This approach to maintenance represents a sophisticated understanding of how AI can contribute to software development: not just writing new features, but continuously improving and maintaining existing code.
The presentation acknowledges that the software factory creates new bottlenecks and scaling challenges. CI capacity is under increased strain from the volume of code changes, requiring anticipation and foundational investments. The number of experiments that can feasibly run simultaneously is limited, creating another bottleneck as more teams want to run A/B tests. Decision-making has shifted from “can we build this?” to “should we build this?”, suggesting that the constraint is no longer development velocity but rather prioritization and strategic choices.
These bottlenecks are honest acknowledgments that scaling AI-powered development isn’t simply a matter of adding more LLM calls. The entire surrounding infrastructure—CI systems, experimentation platforms, and organizational decision-making processes—must evolve to handle the increased throughput. This is a valuable reality check on the narrative that AI will simply make everything faster without requiring corresponding infrastructure investments.
The scale of Uber’s deployment is substantial: 800+ projects using the model gateway, 100 million model requests per day, 1,000+ MCP tools, 2,500 skills with 20,000 executions per day, 40 million entries in the context graph, and 20,000 Cortana sessions per day. These numbers suggest this is a mature production deployment rather than an experimental rollout.
However, the headline metrics—70% of PRs from agents, doubled lines of code per engineer—require careful interpretation. These are quantity metrics that don’t directly translate to business value. More code isn’t necessarily better if it introduces bugs or technical debt. More PRs could actually represent a problem if they require more review time or create integration issues. The focus on validation, quality gates, and continuous evaluation suggests Uber is aware of these concerns and is actively addressing code quality, but the presentation doesn’t provide quality metrics to complement the volume metrics.
This case study represents one of the most comprehensive production deployments of agentic AI for software development publicly discussed. Several LLMOps patterns emerge that are likely applicable to other organizations:
The centralized gateway pattern (for both models and tools) enables consistent policy enforcement, cost attribution, and optimization that would be impossible with decentralized access. The investment in middleware that runs under 100ms latency constraints shows that real-time performance matters even when calling external APIs. The evolution of token optimization strategies demonstrates that even as model costs decline, optimization remains important at scale.
The context graph approach represents a middle ground between fully manual information retrieval and pure vector-based RAG. By investing in structured knowledge representation, Uber gains more predictable and reliable results, but at the cost of engineering effort to build and maintain the graph. The tradeoff between investment and reliability likely makes sense at Uber’s scale but might not be justified for smaller organizations.
The validation strategy with inner loop and outer loop checks reflects hard-won lessons about autonomous code generation. Early enthusiasm about fully autonomous coding has been tempered by the realization that validation is essential, but Uber has avoided the opposite extreme of requiring human review for everything. The multi-layer validation with automated checks followed by human oversight represents a pragmatic middle ground.
The managed marketplace pattern for skills addresses governance challenges that emerge as AI adoption scales. Without centralized management, organizations end up with duplicate efforts, variable quality, and poor discoverability. However, managing 2,500 skills with proper quality gates and continuous evaluation represents significant ongoing investment.
The acknowledgment of new bottlenecks is particularly valuable. Organizations adopting agentic AI should plan for increased load on CI systems, experimentation platforms, and decision-making processes, not just on LLM APIs. The constraint on feasible experiments suggests that even with faster development, other organizational processes become limiting factors.
Overall, while the presentation naturally emphasizes successes and impressive metrics, the level of infrastructure investment, the sophisticated validation strategies, and the honest discussion of bottlenecks suggest a mature understanding of what’s required to deploy agentic AI at production scale. The emphasis throughout on managed, centralized infrastructure rather than ad-hoc adoption by individual teams represents a deliberate architectural choice that enables governance and optimization but requires significant platform engineering investment.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Cursor, a developer tool company, shares their journey of building what they call a "software factory" where AI agents handle increasingly autonomous software development tasks. The presentation outlines how they progressed through levels of autonomy from basic autocomplete to spawning hundreds of agents working asynchronously across their codebase. Their solution involves establishing guardrails through rules that emerge dynamically, creating verifiable systems with automated testing, and building skills and integrations that enable agents to work independently. Results include engineers managing fleets of agents rather than writing code directly, with some features being developed entirely by agents from feature flagging through testing to deployment, though significant work remains in observability, orchestration, and preventing agents from going off-track.
Notion, a knowledge work platform serving enterprise customers, spent multiple years (2022-2026) iterating through four to five complete rebuilds of their agent infrastructure before shipping Custom Agents to production. The core problem was enabling users to automate complex workflows across their workspaces while maintaining enterprise-grade reliability, security, and cost efficiency. Their solution involved building a sophisticated agent harness with progressive tool disclosure, SQL-like database abstractions, markdown-based interfaces optimized for LLM consumption, and a comprehensive evaluation framework. The result was a production system handling over 100 tools, serving majority-agent traffic for search, and enabling workflows like automated bug triaging, email processing, and meeting notes capture that fundamentally changed how their company and customers operate.
