OpenAI's RL and agent infrastructure team designed a comprehensive sandbox cloud system to securely execute untrusted code generated by LLMs in products like ChatGPT and Codex at massive scale. The problem addressed is that modern AI models need to execute code to solve mathematical, programming, and other verifiable reward tasks, but doing so safely requires sophisticated isolation mechanisms. The solution evolved from basic container approaches through user-space kernels to hardware-based virtualization using microVMs with Rust-based VMMs like Cloud Hypervisor and CrosVM. They implemented sophisticated disk persistence through incremental snapshotting at the block level, enabling checkpoint-restore capabilities for long-running agent tasks. The orchestration layer intelligently routes sandboxes across global clusters based on snapshot locality and resource availability, achieving both low-latency creation and high reliability for production AI agent workloads.
This case study from OpenAI’s RL and agent infrastructure team provides comprehensive insights into building production-grade sandbox infrastructure for AI agents that execute untrusted code. The presentation, delivered by Abhishek from OpenAI, covers the evolution from basic fork-exec models through containers and user-space kernels to hardware-based virtualization, along with sophisticated persistence and orchestration mechanisms needed to run AI agents securely at ChatGPT and Codex scale.
The fundamental problem OpenAI faced is that large language models like ChatGPT, while excellent at generating human-like responses, struggle with tasks requiring precise computation or verifiable answers. For example, while models can answer “3 + 3” correctly because this appears frequently in training data, they fail on questions like “how many r’s in strawberry” because such specific queries are less common. The key unlock was providing models with tool-calling capabilities, particularly code execution, which dramatically improves performance on mathematics, programming, and other domains with verifiable rewards.
This capability is essential for both training and production. In training loops, the model receives tasks, generates responses that may include code execution requests, a harness executes the code, and a grader verifies correctness. The training loop then backpropagates to adjust model weights, teaching the model both when to call tools and how to write code that solves problems. On the product side, the same pattern exists without the training loop - the harness parses model responses, executes tools and code, and returns results to continue the interaction.
The presentation grounds its security discussion in Linux fundamentals. In Linux, threads are the smallest unit of execution, and the kernel provides privileged access through system calls. The processor has different execution rings, with kernel mode running in ring 0 with highest privilege and user mode in ring 3. There are two primary attack vectors: gaining root access (staying in ring 3 but becoming the most privileged user) and exploiting the kernel directly to execute code in ring 0, which provides complete system control.
The speaker emphasizes that executing untrusted AI-generated code creates real security risks. While models are generally trained to generate non-malicious code, good security hygiene requires protecting the execution environment - whether a user’s laptop running Codex or a cloud node running ChatGPT. Attacks could be intentional or unintentional; models might become overzealous in trying to help users and attempt to gain elevated privileges to complete tasks. Furthermore, in multi-tenant cloud environments, compromised sandboxes could potentially access other users’ data.
The simplest approach would be an API server that forks a process for each tool call and executes the requested code. This provides native performance but has critical flaws. First, the forked process can directly communicate with the kernel, enabling both root privilege escalation and kernel exploits. Second, there’s a noisy neighbor problem - malicious code with an infinite fork loop could exhaust node resources and prevent other workloads from running.
Containers provide the next level of sophistication through two key Linux primitives: namespaces and cgroups. Namespaces isolate resources - for example, a PID namespace makes processes inside a container appear to have PIDs 1, 2, 3 while they have different PIDs when viewed from the host. Mount namespaces allow mounting filesystems within containers without affecting host visibility. Network, user, and other namespaces provide similar isolation.
Cgroups address the resource consumption problem by controlling how much CPU, memory, and other resources a container can consume, preventing individual containers from bringing down entire nodes. However, the fundamental limitation is that containers are still native host processes. While namespaces prevent containers from attacking each other by isolating resources, processes within containers can still exploit kernel vulnerabilities to gain root access or execute kernel exploits, potentially accessing data from other sandboxes or taking control of nodes.
The attack surface can be reduced through mechanisms like seccomp, which filters available system calls and their arguments. This reduces the kernel attack surface but creates product challenges - it’s often impossible to know in advance what system calls agent workloads might need. Blocking legitimate requests creates poor user experiences, while the feedback loop to adjust seccomp filters is slow. For products like Codex where agents should be able to do “magical things” with Linux systems, overly restrictive filters are problematic.
gVisor represents an intermediate approach that reimplements much of the Linux kernel in user space. Its key security story centers on reducing kernel API exposure. The “Sentry” component is a user-space kernel written in Go that implements the Linux API including process and workload management. When applications make system calls, they’re intercepted and serviced by this user-space program rather than the host kernel. File system access goes through another daemon called the “Gofer.”
The security advantage is that exploits target user-space code running in ring 3 rather than kernel code in ring 0. However, the Sentry and Gofer themselves run atop the host kernel, so chained exploits remain possible - attackers could first exploit vulnerabilities in the Sentry or Gofer, then exploit from those components to the host kernel. With increasingly sophisticated AI models, the possibility of chaining exploits through automated vulnerability discovery becomes more realistic. While the chain is harder than direct kernel exploitation, the host kernel remains reachable.
Hardware-based virtualization provides the strongest security boundary by leveraging CPU-level abstractions. Even if untrusted code gains root access or exploits the guest kernel to reach ring 0, the host remains protected through hardware isolation. This works because the guest kernel runs in ring 0 within a separate processor context called VMX non-root mode, while the host kernel and hypervisor run in ring 0 within VMX root mode.
This gives the guest kernel complete control within its own context but no control over the host. The processor must switch between guest and host contexts when guests access privileged resources, which introduces a performance penalty. However, the speaker argues that “security tricks can cover performance issues, but they cannot hide security breaches” - companies can lose trust once and find it very hard to regain, making security the higher priority.
The presentation describes what the speaker calls “the seven stages of sandboxing” - companies typically try containers, gVisor, V8 isolates, and other approaches before eventually adopting VMs because they need full Linux functionality and strong security. The advice to startups is to use microVMs from the start to avoid two years of migration.
The Virtual Machine Monitor (VMM) is software that manages virtual machines by interfacing with the Linux hypervisor API at /dev/kvm. Traditional VMMs like QEMU allocate memory, set up kernels and root filesystems, and call into this API. Inside the guest, devices like block storage and network appear as regular Linux devices, but when accessed, they exit to the host context where device backends running in the VMM process emulate the hardware.
Paravirtualization achieves good performance by making guest drivers aware they’re running in virtual environments. They communicate via virtio, a more efficient protocol than fully emulated hardware. Devices appear as PCI devices to the guest, but operations efficiently exit to the host only when necessary rather than on every filesystem operation. From the host perspective, these appear as regular threads - a block I/O thread from the guest context wakes up in the VMM process when it exits.
A significant shift occurred in 2023 with new Rust-based VMMs emerging as alternatives to QEMU. While QEMU supports many architectures and devices, this comprehensiveness brings bloat, and being written in C, it has historically been targeted by escape attacks exploiting device implementations. CrosVM was the first Rust-based VMM, developed by a team at Google (where the speaker previously worked) to support Linux VMs on Chromebooks.
The advantages of Rust-based VMMs like CrosVM, Firecracker, and Cloud Hypervisor include memory safety eliminating entire classes of vulnerabilities, and device jailing where individual devices receive access only to their relevant resources. If an attacker compromises the block device, they can’t access network resources, providing defense in depth. The “micro” in microVMs refers not to what runs inside but to the VMM itself - these have much smaller memory footprints and boot faster due to supporting fewer devices and having less bloat.
Firecracker, used by AWS Lambda and serverless infrastructure, forked from CrosVM. Cloud Hypervisor is a more general-purpose VMM with contributions from multiple companies. The speaker’s team uses Cloud Hypervisor, which exposes APIs over Unix domain sockets. Starting a microVM involves forking the Cloud Hypervisor binary, calling its create API with root filesystem, kernel, CPU and memory specifications, then calling start, which invokes /dev/kvm to launch the guest.
Communication between the host harness and guest sandbox occurs through various mechanisms. The guest typically runs a PID 1 process exposing an API server for operations like saving state or attaching devices. This can use vsock (a socket type for guest-host communication) or the node’s IP stack.
MicroVMs provide excellent hardware-level isolation with significantly harder (though not impossible) attack chains requiring exploitation of the KVM stack and then device layers. Devices can be jailed using seccomp and other hardening, limiting blast radius from compromised devices. The performance overhead from context switching between guest and host is the primary cost.
Memory management is more complex than with containers - a balloon driver must request the guest to return memory rather than allowing immediate reclamation. GPU access presents challenges: virtio-gpu provides high-level graphics library access, but direct metal access requires VFIO, which can only serve one sandbox at a time rather than supporting multi-tenancy. Despite these limitations, the security benefits typically outweigh the operational complexity.
The speaker emphasizes that while giving models compute capabilities was the first unlock, durable storage represents the next frontier. Running sandboxes without persistent disks is like giving someone a computer that loses all data when closed - not practical for real work. As agent tasks become more complex and longer-running, with users creating presentations and GitHub repositories within sandboxes, data loss from node failures or model flakes wastes GPU tokens and degrades user experience.
Persistence unlocks three major capabilities. First, it improves reliability and scale - counterintuitively, persistence helps with these seemingly orthogonal concerns. For long-running tasks with many installed packages and created artifacts, periodic checkpointing allows restoring sandboxes to exact checkpoint states on different nodes after failures. This also enables intentional migration for cluster upgrades or A/B testing.
Second, it supports truly long-running tasks. The speaker mentions running tasks for three days in Codex’s “gold mode,” and expects this trend to continue in cloud environments. Checkpointing provides resilience against infrastructure failures for such workloads. Third, persistence enables exploration of multiple solution paths through Monte Carlo tree search-style approaches. Harnesses can checkpoint states, explore different branches, backtrack, and checkpoint again, allowing rollouts over many days to find optimal solutions. The speaker expresses hope that with solid primitives, this could help solve diseases and discover new drugs by enabling models to work on problems for extended periods.
At ChatGPT and Codex scale, snapshotting solutions must support incremental snapshots - capturing only differences between successive snapshots rather than gigabytes of full data repeatedly. The snapshotting API itself must be cheap and fast so models and harnesses can snapshot frequently while exploring. Restoration must be equally fast for good product experiences, essentially treating restore as creation from a snapshot.
Two paradigms exist: always-on saving where harnesses don’t explicitly call save APIs, and explicit saving where harnesses control snapshotting. Design choices include whether to do incremental versus full snapshots, whether to snapshot entire root filesystems or specific folders like workspaces, and whether to operate at the filesystem level (entire files) or block level (changed blocks only). The general flow involves determining what changed, compressing it, storing to cloud storage, then downloading and restoring when needed.
Linux represents disks as block devices with logical blocks from zero to end. Filesystems map directories and files into inode data structures that specify which logical blocks store file data at different offsets. Firmware on the physical disk maps logical blocks to actual sectors or pages. This hierarchy enables efficient block-level snapshotting.
For explicit persistence, the implementation uses copy-on-write filesystems like XFS. Starting from a base image (like a Codex or ChatGPT base), a zero-latency copy creates a writable layer. When blocks change, they’re modified in this layer. On snapshot requests, fiemap determines which blocks and ranges changed, these are compressed and stored to cloud storage. The implementation can return from the snapshot call immediately while uploading in the background, minimizing latency perceived by the harness.
Restoration downloads the diff artifact, determines changed extents, applies them atop the base image, and starts the microVM with the restored state. This provides exact block-level state restoration while minimizing data transfer through incremental diffs.
For always-on persistence, the solution implements a filesystem atop object storage like GCS or S3, using Network Block Device (NBD) protocol. From within the sandbox, this appears as a regular block device, but underneath operates a tiered cache architecture. Blocks are cached first in an in-cluster cache, which writes back to object storage. This provides a globally consistent, performant filesystem within microVMs with POSIX compliance.
POSIX compliance matters because models are trained extensively on standard POSIX systems and work best with familiar semantics. Alternative approaches like NFS lack performance and full POSIX compliance, making them less suitable despite being simpler to deploy.
The speaker emphasizes that storage persistence is critical for advancing agent capabilities. The ability to snapshot and restore quickly enables harnesses to recover from failures and explore solution spaces through tree search and backtracking. This transforms agents from ephemeral code executors to persistent knowledge workers that can tackle longer-horizon tasks with resilience.
Running sandboxes reliably at scale requires orchestrating across many nodes worldwide. While the speaker avoids diving into Kubernetes details, keeping the discussion at first principles, the architecture groups nodes into clusters spread across regions. A top-level control plane selects clusters based on region load and other factors, ideally choosing clusters geographically close to ChatGPT clusters for low-latency harness communication.
Within clusters, a scheduler determines which specific node should run each sandbox based on load, health, and other factors, avoiding failed or dying nodes. Low latency and reliability remain the North Star metrics for the architecture.
Several microVM features enable low-latency sandbox creation. Many systems “cheat” by pre-warming sandbox pools and selecting from them on demand, which works but consumes resources. An alternative approach leverages microVM memory snapshots to start sandboxes just-in-time in milliseconds as requests arrive. A hybrid solution maintains a warm pool but grows it from memory snapshots, getting the best of both approaches. The trade-off is between consuming CPU and memory in idle states versus achieving the lowest possible latency.
The orchestration layer can leverage snapshot lineage information for intelligent routing. Since snapshots form chains of layers, when restoring from a snapshot requires downloading multiple layers, the scheduler can route to nodes that already have relevant layers cached. In a scenario where different nodes have cached different subsets of a snapshot’s layer lineage, the scheduler assigns the highest score to nodes with complete or near-complete layer sets, minimizing download time and accelerating restoration.
This snapshot-aware routing integrates persistence with orchestration, improving both creation speed and overall reliability by making restoration more efficient and predictable.
The presentation distinguishes between research/training needs and product requirements. In research, throughput is paramount - running many training loops at scale with many rollouts (different solutions to the same task) in parallel. Taking many shots on goal quickly accelerates training.
In product, latency dominates. Successful products of the last 20 years have been fast, and if sandboxes don’t start quickly and execute code rapidly, users churn. The speaker notes that reliability is critical on both sides - failed sandboxes waste GPU tokens, which are extremely valuable, whether during training or production. Similarly, security matters equally for both - compromised training infrastructure could lead to model weight exfiltration or infrastructure attacks, while compromised product sandboxes could expose user data or attack infrastructure.
The presentation references the phenomenon of products like Cursor’s Codex running locally on users’ laptops, calling it “a slap in the face for 20 years of cloud computing.” The speaker views this as a peek into the future but hopes that agents will run in the cloud where they can be persistent and long-running, rather than requiring users to keep laptop lids open to prevent agent sleep. There’s mention of people renting VPSs and running services on providers like Hetzner or Mac minis, indicating demand for cloud-based agent infrastructure.
This underscores the market need for robust sandbox clouds - if individuals are going to these lengths to run agents persistently, there’s clear value in professional, secure, scalable infrastructure that handles these needs properly.
While the presentation provides valuable technical depth from a team actually operating this infrastructure at scale, several caveats deserve consideration. The security claims, while well-reasoned, present microVMs as significantly more secure than alternatives, but the speaker acknowledges that attacks on KVM and device layers have occurred and will likely continue. With sophisticated AI models potentially helping discover and chain exploits, the security advantage may narrow over time.
The performance trade-offs of microVMs compared to containers or gVisor are mentioned but not quantified. Organizations need concrete latency and throughput numbers to make informed decisions. The suggestion to “just use microVMs from the start” may not suit all use cases - simpler approaches might suffice for lower-risk scenarios with less sensitive data.
The persistence architecture, while sophisticated, introduces complexity and dependencies on cloud storage with associated costs and potential failure modes. The always-on persistence approach with distributed block storage adds multiple caching layers that could introduce bugs or performance issues. The presentation doesn’t deeply address failure scenarios in the persistence layer.
On orchestration, the snapshot-aware routing is clever but assumes snapshot locality is the dominant factor in placement decisions. In practice, compute resources, network topology, data gravity from other services, and cost considerations all play roles. The presentation’s first-principles approach intentionally avoids these messy realities but they matter greatly for production deployments.
Finally, the framing around “security tricks versus security breaches” creates a somewhat false dichotomy. Defense in depth involves multiple layers, and dismissing approaches like containers with seccomp as insufficient doesn’t account for defense-in-depth strategies that might combine multiple isolation mechanisms. That said, for OpenAI’s threat model with potentially adversarial or misaligned model behavior, the conservative approach toward VM-based isolation makes sense.
Despite these considerations, the case study provides genuine insights into production LLMOps challenges for code-executing agents. The evolution from simple fork-exec through containers and user-space kernels to hardware virtualization reflects real security learning. The emphasis on persistence as an unlock for longer-horizon tasks and exploration is forward-looking and likely to influence how other organizations build agent infrastructure. The integration of checkpointing with orchestration through snapshot-aware routing demonstrates sophisticated systems thinking about how components interact across the stack.
Notion, a knowledge work platform serving enterprise customers, spent multiple years (2022-2026) iterating through four to five complete rebuilds of their agent infrastructure before shipping Custom Agents to production. The core problem was enabling users to automate complex workflows across their workspaces while maintaining enterprise-grade reliability, security, and cost efficiency. Their solution involved building a sophisticated agent harness with progressive tool disclosure, SQL-like database abstractions, markdown-based interfaces optimized for LLM consumption, and a comprehensive evaluation framework. The result was a production system handling over 100 tools, serving majority-agent traffic for search, and enabling workflows like automated bug triaging, email processing, and meeting notes capture that fundamentally changed how their company and customers operate.
This case study examines Cursor's implementation of reinforcement learning (RL) for training coding models and agents in production environments. The team discusses the unique challenges of applying RL to code generation compared to other domains like mathematics, including handling larger action spaces, multi-step tool calling processes, and developing reward signals that capture real-world usage patterns. They explore various technical approaches including test-based rewards, process reward models, and infrastructure optimizations for handling long context windows and high-throughput inference during RL training, while working toward more human-centric evaluation metrics beyond traditional test coverage.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
