Warp, a developer tools company that evolved from a terminal application into an agentic development environment, built a cloud agent platform to enable AI agents to perform long-running, complex development tasks beyond what's possible on local machines. The platform abstracts infrastructure complexity by providing flexible sandboxes (self-hosted and managed), multi-harness support for different AI models, agent orchestration capabilities, and comprehensive APIs that enable both technical and non-technical team members to build custom automation workflows. When Warp open-sourced their codebase, they deployed agents to manage the influx of issues and PRs, automating triage, specification drafting, implementation support, and code review, which allowed human reviewers to focus only on high-quality contributions that had already passed agent-based review gates.
Warp’s cloud agent platform represents a comprehensive approach to deploying AI agents in production for software development workflows. The company, which initially built a terminal application, evolved into creating a full agentic development environment. The speaker, who has eight years of experience building developer tooling including work on Jupyter Notebook and at Microsoft, presents the engineering philosophy and technical architecture behind their cloud agent platform. The central thesis is that great developer tools must meet developers where they are and grow with them, adapting to individual workflows while abstracting away infrastructure complexity.
The fundamental challenge they addressed was extending agentic coding patterns beyond local machines to the cloud, where agents could handle more long-running and resource-intensive tasks. However, moving to the cloud introduces significant infrastructure complexity, and the core design principle guiding their platform is that platforms should absorb complexity before it reaches users.
The platform’s architecture is built around several key primitives that each embody the philosophy of hiding complexity while providing flexibility.
Sandboxes and Compute Management: The first major component is the sandbox environment where agents execute their work. Initially, Warp provided self-hosted sandboxes to create an easy on-ramp for users, but they recognized that teams doing serious work typically manage their own infrastructure and have specific security requirements, deployment practices, and organizational preferences. To accommodate this reality, they added support for both managed hosting and self-hosting options. This allows teams to run agent workloads on their own infrastructure while maintaining a consistent interface and behavior model. The platform abstracts the complexity of managing these different hosting modes from the user’s perspective.
Multi-Harness Support: A critical design decision was supporting multiple AI harnesses rather than locking users into a single model or provider. The team recognized the diversity in developer preferences and the practical need to use different tools for different jobs. During the presentation, the speaker polled the audience about preferences between Claude, Codex, and other harnesses, illustrating the heterogeneity in the field. The platform provides structure and guardrails around harness integration to ensure consistency across different models. This means that regardless of whether users are working with Claude, Codex, or custom harnesses, the experience for storing conversation state, rehydrating sessions, and interacting with artifacts remains uniform. The artifacts produced by agents—such as pull requests, issues, and generated files—are structured consistently regardless of which harness generated them.
Agent Orchestration: The platform recognizes that real engineering work rarely fits within a single prompt or agent interaction. Complex software development tasks often require multiple specialized agents working in concert. For example, a typical workflow might involve one agent researching a problem and planning a solution, another implementing it, and a third validating the implementation. Each of these agents might use different harnesses and models to create an adversarial and robust approach to problem-solving.
Warp supports two primary modes of orchestration. The first is prompt-based orchestration, where a user can issue a single prompt that triggers an orchestrator agent to handle the complexity of delegating work to subagents, mediating messages between them, and tracking progress. The second mode is API-based orchestration, where developers can programmatically fire off requests to spin up subagents attached to parent agents with specific configurations. This API-driven approach is fundamental to the platform’s extensibility and enables composability beyond the constraints of any single UI.
A defining characteristic of Warp’s cloud agent platform is its API-first approach. Every component of the surface area is exposed through APIs, including agent and subagent creation, environment and compute management, and artifact manipulation. This design decision reflects a deep understanding that great platforms enable users to build on top of them rather than constraining them to predetermined use cases or user interfaces.
The composability enabled by this API approach has had significant internal impact at Warp. Non-engineering team members have been able to use the SDK and APIs to build custom automation tools. The developer relations team, for instance, built tooling to manage social media mentions across Twitter, Reddit, and other platforms. Their system uses agents to pick up incoming mentions, perform sentiment analysis, understand user intent, and propose responses that the social media team can use to engage with the original posts. Other internal use cases include answering product queries and conducting competitive research. This democratization of agent-based automation to non-technical team members validates the platform’s abstraction of complexity.
The most substantial validation of the platform came when Warp open-sourced their codebase approximately three months before this presentation. The open-sourcing was highly successful, with GitHub stars increasing from around 20,000 to over 60,000, thousands of incoming PRs, and hundreds of new contributors. This scale of engagement created a significant operational challenge that the team addressed by deploying their agent platform to manage the repository itself.
The team’s philosophy was to enable agents to participate meaningfully in the software development lifecycle without simply having them autonomously write and submit code. Instead, agents were integrated into structured processes alongside human contributors. When new issues are filed in the Warp repository—whether bug reports or feature requests—an agent automatically begins triaging the issue. The agent performs research across the codebase and contextual information in the repository to understand the proposal. If the initial query is abstract or lacks sufficient detail, the agent asks clarifying questions to gather more information. This addresses a perennial challenge in open source projects where contributors submit issues without adequate detail, making it difficult to move work forward.
Beyond triage, agents help draft initial specifications and implementation plans, support the implementation process, and provide a review gate for all contributed PRs. Critically, human reviewers are not pinged until an agent has approved a PR through multiple review iterations. This filtering mechanism allows human reviewers to focus exclusively on high-signal, high-quality contributions rather than being overwhelmed by the volume of submissions. From thousands of PRs, only those that meet agent-defined quality standards reach human attention.
An important principle in Warp’s approach is the implementation of self-improvement loops. As more PRs are submitted and more code examples accumulate in the repository, the agents improve their performance. This continuous improvement mechanism enhances the overall software development lifecycle. The system is designed to be observable, allowing the team to inspect and understand how agents are performing and making decisions. This observability is essential for refining the process over time—the platform is not static but reacts to usage patterns and evolving requirements.
The presentation emphasizes the importance of cost-effectiveness in production agent systems. Using the metaphor of a pottery workshop, the speaker notes the need to reduce the number of defective outputs—broken mugs in pottery, buggy software in development. This requires balancing quality with token consumption and computational costs. The platform’s design considers these economic constraints while maintaining the quality of outputs.
The speaker explicitly pushes back against the common industry term “software factory” for automated development systems, preferring the metaphor of a “workshop.” This is not merely semantic but reflects a different conceptual model. The speaker shares a story about purchasing a handcrafted mug from a potter at a farmer’s market. The potter had meticulously designed every detail—the curve of the handle, a thumb rest dimple, overflow-catching glazing—and had also developed a sophisticated workshop with different stations for different components, processes for sourcing and preparing materials, and verification steps for quality control. This workshop enabled dozens of apprentices to produce hundreds of handcrafted mugs daily.
The workshop metaphor emphasizes several key points: workshops are serious and repeatable systems for turning ideas into reality; they are malleable and react to feedback; they maintain close interaction loops between humans and the spaces where they work; and they expand the definition of who can be a builder. This philosophy directly informs the technical design of Warp’s platform—exposing primitives, supporting automations that react to real-world events, ensuring observability, enabling continuous improvement, and maintaining cost-effectiveness.
From an LLMOps perspective, several aspects of Warp’s approach merit balanced consideration. The platform’s strength lies in its abstraction of infrastructure complexity and its flexibility in supporting different hosting models and harnesses. This flexibility is crucial for production adoption, as enterprises have diverse security, compliance, and operational requirements. The API-first design is a best practice that enables extensibility and prevents vendor lock-in to specific workflows.
However, the presentation is essentially promotional in nature, coming from a company building and selling this platform. Claims about effectiveness—such as agents successfully managing thousands of PRs or non-technical team members building sophisticated automation—are presented without quantitative validation. There are no metrics provided on agent accuracy rates, false positive rates in code review, time savings, or cost comparisons. The self-improvement mechanism is mentioned but not detailed in terms of how it’s implemented, what feedback signals are used, or how quickly improvements materialize.
The multi-agent orchestration capability, while powerful, introduces complexity in terms of managing state across agents, handling failures and retries, and ensuring consistent behavior when different harness models may have varying capabilities and costs. The presentation doesn’t address error handling, debugging multi-agent workflows, or how conflicts between agents are resolved.
The claim that human reviewers only see PRs after agent approval is compelling for managing scale, but it also raises questions about what types of issues agents might miss and whether over-reliance on agent filtering could inadvertently exclude valuable but unconventional contributions. The balance between automation and human judgment in creative processes like software development remains an open question.
The cost-effectiveness mentioned as a principle lacks concrete implementation details. Token costs for multi-agent workflows with multiple harnesses can escalate quickly, especially with iterative review processes. The presentation doesn’t address cost monitoring, budgeting, or optimization strategies in production deployments.
Overall, Warp’s cloud agent platform represents a thoughtful approach to production LLM deployment for software development workflows, with particularly strong design around flexibility, composability, and abstraction. The proof points from their open source repository management and internal use cases are promising, though the full operational picture including failure modes, costs, and limitations remains less clear from this presentation. The philosophical framework of building “workshops” rather than “factories” reflects a human-centered approach to AI-augmented development that prioritizes empowering diverse builders rather than fully automating them out of the process.
Notion, a knowledge work platform serving enterprise customers, spent multiple years (2022-2026) iterating through four to five complete rebuilds of their agent infrastructure before shipping Custom Agents to production. The core problem was enabling users to automate complex workflows across their workspaces while maintaining enterprise-grade reliability, security, and cost efficiency. Their solution involved building a sophisticated agent harness with progressive tool disclosure, SQL-like database abstractions, markdown-based interfaces optimized for LLM consumption, and a comprehensive evaluation framework. The result was a production system handling over 100 tools, serving majority-agent traffic for search, and enabling workflows like automated bug triaging, email processing, and meeting notes capture that fundamentally changed how their company and customers operate.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Stripe, processing approximately 1.3% of global GDP, has evolved from traditional ML-based fraud detection to deploying transformer-based foundation models for payments that process every transaction in under 100ms. The company built a domain-specific foundation model treating charges as tokens and behavior sequences as context windows, ingesting tens of billions of transactions to power fraud detection, improving card-testing detection from 59% to 97% accuracy for large merchants. Stripe also launched the Agentic Commerce Protocol (ACP) jointly with OpenAI to standardize how agents discover and purchase from merchant catalogs, complemented by internal AI adoption reaching 8,500 employees daily using LLM tools, with 65-70% of engineers using AI coding assistants and achieving significant productivity gains like reducing payment method integrations from 2 months to 2 weeks.
