## Overview Cursor, an AI-powered code editor company, developed Composer, an agent-based LLM specifically designed for coding tasks that represents a significant advancement in production LLM deployment for software engineering. The case study, presented by Sasha Rush from Cursor's AI research team, details the comprehensive LLMOps infrastructure and methodology used to train and deploy a model that achieves near-frontier-level performance while being four times more efficient at token generation than comparable models. The motivation for building Composer stemmed from the success of Cursor Tab, a popular feature in their editor that users found "delightful" due to its fast, smart responses. The team recognized that speed wasn't just about raw performance metrics—it fundamentally changed the user experience by allowing developers to maintain their chain of thought and remain in flow. They built a prototype called "cheetah" that demonstrated this fast agentic experience, and user feedback describing it as "alien technology" validated their hypothesis that combining intelligence with speed could create a differentiated product experience. ## Problem Definition and Goals The Cursor team set out with a dual objective: build a model that was both highly intelligent for realistic coding work and felt very fast in practice. Importantly, they weren't targeting arbitrary benchmarks but rather focusing on what matters for day-to-day software engineering: the ability to work with large codebases and maintain adherence to codebase standards. They built an internal benchmark from their own repositories specifically to measure these factors. Speed was defined holistically—not just token generation efficiency, but also how quickly the model could produce edits in the editor and leverage capabilities like parallel tool calling to deliver fast results. This multi-dimensional approach to both intelligence and speed required rethinking traditional LLM training and deployment paradigms. ## Agent Architecture and Tool Space At its core, Composer operates as an agent that interacts through a tool space. When a user submits a query to the Cursor backend, the agent reads the query and makes a series of tool calls. The team designed approximately 10 tools for production use, including read file, edit file, codebase search, lint collection, and terminal command execution. Critically, the agent can call these tools both serially and in parallel, a capability that proved essential for the fast user experience. From a low-level perspective, the agent is still a large language model generating tokens, with some tokens forming XML patterns that enable tool calls and their arguments. However, from a reinforcement learning perspective, the team conceptualized the action space as combinatorial tool calls rather than individual tokens. This abstraction proved important for training efficiency and reasoning about agent behavior. The frontend experience reflects this tool-based architecture: users see summaries of read operations, real-time edits as they're made, and both the terminal commands and their outputs. This transparency helps users understand what the agent is doing and builds trust in the system. ## Reinforcement Learning Training Approach The core training methodology involves agent-based reinforcement learning that runs as close to the production Cursor environment as possible. Training data is treated as user queries sent to the model, which then calls tools to accomplish goals. The key innovation is running many rollouts simultaneously in parallel—effectively running many instances of Cursor at the same time—and scoring the outputs to determine which tool-calling strategies work better. This approach seems conceptually simple, but the implementation challenges are substantial. The team identified three core scaling challenges: **Train-inference matching**: Training a mixture-of-experts language model distributed across thousands of GPUs is already difficult for pre-training or supervised fine-tuning, but reinforcement learning doubles the complexity because both training and sampling versions must work in sync. **Realistic rollout complexity**: Modern agent rollouts use 100,000 to 1 million tokens and make hundreds of tool calls. Different rollouts produce varying numbers of tool calls and take different amounts of time, creating significant synchronization and efficiency challenges. **Production consistency**: The team's goal of training through their production product meant using exactly the same tool formats and responses as the production Cursor agent, but at much larger scale. Notably, the solutions to these machine learning challenges were all infrastructure-based, highlighting how critical engineering and systems work is to successful LLMOps at scale. ## Infrastructure Architecture The training infrastructure consists of three primary server types working in concert: **Trainer**: Uses PyTorch and resembles a standard machine learning stack, but scaled to very large degree. The team developed custom kernels for low-precision training using MXFP8 (microscaling FP8 format), which provides the benefits of FP8 precision with additional scaling factors for better precision and higher quality training. For mixture-of-experts layers on Blackwell chips, this approach yields a 3.5x speedup. Critically, this low-precision training also enables efficient sampling without post-training quantization, making it easier to ship models to the inference server for rollouts. **Inference server**: Primarily uses Ray for orchestrating rollouts, calling tools, and managing advantages. The main challenge addressed here is the "straggler problem"—because agents can install libraries, run arbitrary terminal commands, and perform unpredictable operations, naive implementations would see 10 rollouts returning at vastly different times. The team solved this using Ray with a single controller interface that enables load balancing across many threads and processes, significantly improving efficiency. **Environment server**: Uses microVMs to spin up stateful environments where file changes can be made, terminal commands executed, and linters run. This essentially runs mini versions of Cursor for each training rollout. ## Production-Training Integration One of the most interesting aspects of Cursor's LLMOps approach is the co-design of product and ML training infrastructure. The team leveraged Cursor's "cloud agents" product—which allows users to run agents offline by spinning up VMs of their environments—as the same infrastructure for RL training. This means the production agent server is identical whether running cloud agents for customers or training the RL model. This approach has significant advantages but also challenges. The workload during peak RL training is much spikier than normal product usage, requiring careful handling of burstiness. The team built custom dashboards (using Composer itself) to monitor backend utilization. However, the benefit of using real production environments rather than mocks is substantial: they can introduce specific tools and train the model to be a "power user" of those tools with confidence that the training environment matches production exactly. A concrete example is their custom embedding model for semantic search. When users use Cursor, all files are indexed to allow natural language queries for finding relevant files. By training Composer with exactly the same semantic search model and structure used in production, the model learned to leverage this tool effectively, particularly important for the parallel tool-calling that enables fast user experiences. ## Model Performance and Characteristics The team reports that Composer scores nearly as well as the best frontier models on their internal benchmark and better than models released the previous summer. It significantly outperforms the best open-source models and models branded as "fast." The four-times token generation efficiency advantage over comparable intelligence models is achieved without sacrificing intelligence, representing a genuine Pareto improvement. Performance improvement during training followed a log-scale compute relationship—as they invested more compute into the RL process, performance on their benchmark increased at a regular rate. The model started around the performance level of the best open-source models in the coding domain, then improved through training to the released model's performance level. This represents encouraging evidence for the scalability of RL for hard, specialized tasks. The training also successfully shaped agent behavior in desirable ways. The model learned to call more parallel tools over time, improving end-to-end user experience speed beyond just token generation. It also learned better agentic behavior patterns: early in training it made too many edits with insufficient evidence, but through RL it learned to read more files and perform more searches before making changes, resulting in more thoughtful and accurate code modifications. ## Evaluation and User Reception Beyond internal benchmarks, the real validation came from user feedback after the release. The team emphasized that the combination of speed and intelligence "unlocks a different sort of coding" where developers get quick results and move to their next problem rather than starting an agent, checking social media, and returning later. This behavioral change represents the product goal they were targeting—maintaining developer flow state. Internally, the Cursor development team uses Composer in their day-to-day work, including building dashboards, backends, and various infrastructure components. This self-hosting and "eating their own dog food" approach provides immediate feedback on model performance and user experience issues. ## Critical Assessment and Tradeoffs While the case study presents impressive results, several aspects warrant balanced consideration: **Infrastructure complexity**: The system requires coordinating three separate server types (trainer, inference, environment) with PyTorch, Ray, and microVMs. This represents significant operational complexity and requires expertise across ML training, distributed systems, and virtualization. Smaller teams or organizations might find this infrastructure burden challenging to replicate. **Custom kernel development**: The low-precision training gains depend on custom kernels for specific hardware (Blackwell chips). This creates hardware dependencies and requires low-level systems programming expertise that may not be available to all teams. The 3.5x speedup is impressive but comes with maintenance and portability considerations. **Evaluation methodology**: The team uses an internal benchmark built from their own repositories. While this makes sense for their use case, it limits external validation and comparison with other approaches. The claimed performance advantages relative to "frontier models" and open-source alternatives would benefit from standardized benchmark results. **Compute requirements**: The log-scale compute investment mentioned suggests very large computational resources were required for training. The case study doesn't provide specific numbers on GPU hours, costs, or energy consumption, making it difficult to assess feasibility for other organizations. **Production environment dependency**: Training through the production environment creates tight coupling between ML training and product infrastructure. While this ensures consistency, it also means changes to product features or infrastructure could require retraining or adjustments to the RL pipeline. **Scalability limitations**: The straggler problem with variable-length rollouts and unpredictable tool execution times represents an inherent scalability challenge for agent-based RL. While Ray helped address this, the solution adds complexity and may have throughput limitations. ## Lessons and Insights The presenter, Sasha Rush, offers several reflections that provide valuable insights for LLMOps practitioners: **RL for specialized models**: The experience suggests reinforcement learning is particularly effective for building targeted models that are extremely smart in customized domains, representing a paradigm shift from general-purpose LLMs. **Infrastructure as research**: The team found that much of their RL work was driven by infrastructure developments rather than pure ML research. This highlights how production LLM systems require integration of product, scale, and ML training—touching all parts of modern software systems. **AI-assisted development**: The development team uses the agents they're building to facilitate their own work, enabling a small team to move quickly. This creates a positive feedback loop where improvements to the model directly improve development velocity. **Product-ML co-design**: The ability to co-design both the product and ML training infrastructure proved valuable for ensuring consistency and enabling features like semantic search to be deeply integrated into agent behavior. ## Technical Debt and Long-term Considerations While not explicitly discussed in the presentation, several long-term considerations are implicit in the approach: **Maintenance burden**: Custom kernels, complex distributed training infrastructure, and production-training integration all represent ongoing maintenance work that will be required as hardware, software dependencies, and product features evolve. **Monitoring and observability**: The team built dashboards for monitoring backend utilization during training, but production monitoring of agent behavior, tool usage patterns, and failure modes likely requires additional observability infrastructure. **Version management**: With tight coupling between training and production environments, managing model versions, rollback procedures, and A/B testing becomes more complex than with stateless inference systems. **Cost optimization**: Running thousands of microVMs for environment simulation during training represents significant infrastructure costs that will require ongoing optimization, especially as training scales further. ## Conclusion Cursor's Composer represents a sophisticated example of modern LLMOps, demonstrating how reinforcement learning can be applied at scale to build specialized, high-performance agent systems. The case study illustrates that achieving production-quality agent-based LLMs requires not just ML expertise but deep infrastructure engineering, careful product-ML integration, and willingness to invest in custom optimization work like kernel development. The dual focus on intelligence and speed, operationalized through internal benchmarks and parallel tool calling, shows thoughtful product thinking about what actually matters for user experience rather than chasing arbitrary metrics. The use of production infrastructure for training ensures consistency but requires careful engineering to handle the scale mismatch between training and normal operations. For organizations considering similar approaches, the key takeaway is that agent-based LLMs in production represent a systems engineering challenge as much as a machine learning one. Success requires expertise across distributed systems, ML training, product engineering, and domain-specific optimization. The results can be compelling—fundamentally changing how users interact with AI systems—but achieving those results demands significant infrastructure investment and cross-functional coordination.