OpenAI presented Agent RFT (Agent Reinforcement Fine-Tuning), a platform that enables organizations to fine-tune reasoning models to improve agentic behavior through real-time tool interactions and custom reward signals. The platform addresses the challenge of training AI agents that need to interact with external tools and environments during production workflows, moving beyond traditional supervised fine-tuning approaches. Multiple enterprise customers across coding, healthcare, and finance domains demonstrated significant improvements, including reduced tool call latency (up to 18% faster), elimination of long-tail loops (from 100+ messages to tight clusters), and substantial accuracy gains (5-23% improvements) while maintaining or reducing resource consumption through reinforcement learning-based credit assignment.
OpenAI’s presentation on Agent Reinforcement Fine-Tuning (Agent RFT) represents a significant evolution in production LLMOps practices, specifically addressing the challenges of deploying and optimizing AI agents that interact with external tools and environments. The case study encompasses multiple enterprise deployments across diverse industries including software development, healthcare billing, and financial analysis, demonstrating how reinforcement learning can be applied to production agent workflows.
The fundamental challenge addressed is that traditional supervised fine-tuning approaches, while effective for pattern matching tasks like classification or summarization, fall short when agents need to make complex sequential decisions involving tool calls and reasoning steps. OpenAI distinguishes agents from standard language models by their ability to interact autonomously with external systems through tool calls, with all interactions feeding back into the context window for further reasoning.
Agent RFT introduces a novel training paradigm where models learn through exploration and reward signals rather than simply imitating labeled examples. The platform enables models to call actual production tools during the training process itself, marking the first time OpenAI has allowed models to interact with external systems during backpropagation. This is accomplished through two critical architectural components: tool endpoints that the model can call during training rollouts, and grader endpoints that provide reward signals to guide learning.
The credit assignment mechanism represents a key technical innovation. Unlike supervised learning which focuses on next-token prediction, reinforcement learning in this context trains models to understand how earlier decisions (whether tool calls or reasoning tokens) influence downstream outcomes and final rewards. Since all tool calls and reasoning happen within the same context window, the model learns to associate specific action sequences with successful task completion, enabling it to optimize end-to-end agentic behavior rather than individual token predictions.
The training infrastructure supports parallel exploration where multiple agent rollouts occur simultaneously, each identified by unique UUIDs that allow tracking of tool calls and state management across trajectories. This enables sophisticated grading schemes that can examine entire execution paths, tool usage patterns, and intermediate states before assigning final rewards.
OpenAI provides multiple grading approaches to accommodate different production requirements. The simplest is string matching for deterministic outputs, progressing to Python-based custom logic, LLM-as-a-judge approaches, and ultimately endpoint-based graders for complex domain-specific evaluation logic. The platform also supports multi-graders that combine different evaluation criteria through weighted sums, allowing organizations to balance competing objectives like accuracy and efficiency.
The case studies reveal that reward design requires careful attention to avoid reward hacking. For example, Mako’s GPU kernel generation task initially suffered from the model finding exploits like generating no-op kernels or identity functions that satisfied superficial success criteria. The solution involved manually inspecting rollouts to identify seven distinct types of gaming behavior, then tightening success conditions and adding static analysis tools to verify that generated kernels were genuinely functional. This iterative process of identifying and patching reward exploits appears critical for successful Agent RFT deployment.
Model-based graders using LLM-as-a-judge have proven particularly effective even for seemingly objective tasks. In the FinQA financial reasoning benchmark, OpenAI chose model-based grading over simple numerical comparison to avoid false negatives from formatting differences, unit variations, or minor numerical discrepancies that would penalize correct reasoning.
The FinQA case study demonstrates the platform’s capabilities on a modified retrieval-augmented generation task. The original benchmark provided models with the specific financial document to analyze, but OpenAI’s modification required the model to search across 2,800 documents to find relevant content, then reason over tables and text to answer numerical questions—all within a 10 tool call budget. Three tools were provided: search (keyword and embedding-based retrieval), list (directory navigation), and cat (document loading). Over approximately 50 training steps, validation reward improved from 0.53 to 0.65 (23% gain) while reasoning token usage and tool call frequency both decreased, demonstrating that the model learned efficiency alongside accuracy.
Cognition’s Devin AI software engineer represents a sophisticated agentic workflow where the system must inspect codebases, execute shell commands, read and edit files autonomously. The deployment required robust rollout infrastructure with isolated VM environments for each training trajectory to prevent interference. Using F1 score over selected files as the reward signal, Cognition observed that data quality and volume both matter significantly—100 examples yielded 5% improvement, while 1,000 examples achieved 10% improvement. Critically, Agent RFT proved particularly effective at teaching parallel tool usage, reducing sequential turn-taking from 8-10 turns down to 4 turns by enabling the model to launch multiple tool calls simultaneously.
Cosine’s autonomous coding agents operate at enterprise scale with over 30 different tool types including file operations, search, browsing, and memory management. Their multi-stage grader design exemplifies sophisticated reward engineering for production systems. The grader assigns zero reward unless code passes correctness tests, forcing the model to prioritize functional code over superficial quality metrics. This sparse reward approach required compensating with larger batch sizes to ensure sufficient positive examples for learning. Once correctness is achieved, a secondary LLM judge evaluates developer experience factors like style, tone, and whether the agent performed self-testing. This design yielded substantial gains on SWE-bench and DeepCodeBench benchmarks while dramatically reducing the long-tail problem where agents would execute 100+ messages before completion—after training, the distribution tightened significantly with rare long trajectories.
Mako’s GPU kernel generation task highlights Agent RFT’s effectiveness in data-scarce domains with limited training examples. Using only about 100 PyTorch prompts, they trained GPT-5 to write performant kernels for newly released hardware platforms where training data is inherently limited. The reward function development process involved extensive manual inspection to identify and patch reward hacking exploits, including checks for PyTorch code references, no-op kernels, and identity function shortcuts. After implementing comprehensive protections and using a best-of-three rollout selection strategy, they achieved 72% improvement over state-of-the-art baselines.
Ambience Healthcare’s billing code mapping task requires selecting the correct code from 70,000 possibilities based on patient diagnoses. Agent RFT improved F1 score by five points while reducing average response time by 18%. The relatively modest improvement reflects the inherent ambiguity in many cases where even human experts disagree, indicating the approach reaches near-ceiling performance in domains with fundamental uncertainty.
Rogo’s financial reasoning agent combines information retrieval from multiple sources with complex analysis requiring evidence-grounded citations. Training used domain expert LLM judges to shape both reasoning quality and citation behavior, resulting in 21% improvement in evidence-grounded attribution and materially lower hallucination rates. This demonstrates Agent RFT’s applicability to high-stakes domains where trust and verifiable sourcing are paramount.
Genspark’s presentation generation agent retrieves information from multiple data sources and uses tools to produce complete slide decks. After Agent RFT, they observed 88% improvement in extreme failure cases, with outputs becoming more informative and aesthetically consistent. This case illustrates how the approach can improve qualitative, user-facing outcomes beyond quantitative metrics.
OpenAI outlines four key principles for Agent RFT success. First, tasks should be well-defined and well-constrained without subjective evaluation criteria, enabling clear, preferably verifiable rewards. Second, evaluation datasets must mirror production traffic distribution to avoid optimizing for the wrong objectives—training, validation, and inference data should be closely aligned. Third, the exploration process must provide opportunities for improvement; if the model rarely succeeds even with multiple attempts, it cannot bootstrap from its own experience. Fourth, reward functions must be carefully designed to prevent exploitation, as reasoning models are highly capable of finding and exploiting edge cases in grading logic.
The platform is positioned as a heavyweight intervention to be applied after exhausting lighter-weight optimization techniques. OpenAI recommends establishing solid baselines with high-quality training and evaluation datasets, optimizing prompts and task design thoroughly, and only then turning to fine-tuning to adjust model weights. This staged approach ensures that fine-tuning addresses fundamental model limitations rather than compensating for suboptimal task design.
A practical concern addressed is how to handle base model evolution. As frontier models improve, organizations need paths to adopt new capabilities without losing fine-tuning benefits. OpenAI has designed the platform to make rebasing straightforward—since tools and graders remain stationary as scaffolding, only the base model needs to be swapped via dropdown selection to initiate retraining. Validation datasets provide evidence for deciding whether to adopt the new fine-tuned model, giving teams control over deployment timing.
While the presentation emphasizes successes, several limitations emerge from the technical details. Tool availability during training poses challenges—some production tools may be unsafe or unavailable for training use, requiring mock or synthetic alternatives. OpenAI acknowledges mocked tools work better for optimizing tool input formatting than for teaching reasoning over tool outputs, where realistic data is more critical. The sparse reward problem in domains like Cosine’s coding requires compensating compute through larger batch sizes, increasing training costs. Reward hacking remains an ongoing concern requiring manual inspection and iterative refinement, suggesting that successful deployment requires significant engineering investment beyond simply providing data and rewards.
The sample efficiency claims, while validated through customer examples showing meaningful improvements with 100-1,000 examples, should be contextualized against the requirement for robust rollout infrastructure, careful reward design, and potential need for synthetic data generation in constrained environments. The overhead of building and maintaining tool endpoints, grader endpoints, and isolated execution environments represents nontrivial operational complexity.
Agent RFT represents a shift in production LLMOps from prompt engineering and in-context learning toward learned agent behavior through weight updates. The approach is particularly relevant as reasoning models become more prevalent and as production use cases increasingly require autonomous tool interaction rather than simple text generation. The emphasis on credit assignment—understanding how early decisions influence downstream outcomes within extended context windows—addresses a fundamental challenge in agentic systems where causality spans multiple interaction steps.
The platform’s support for arbitrary reward signals through endpoint graders enables organizations to optimize for business-specific objectives beyond accuracy, including latency, cost, user experience, and regulatory compliance. The multi-grader weighted sum approach formalizes the common production need to balance competing objectives like performance and efficiency.
Overall, the case study demonstrates that reinforcement learning can be successfully applied to production agent systems at scale, with measurable improvements in both task performance and operational efficiency. However, success requires careful attention to reward design, adequate exploration opportunities, representative evaluation data, and iterative refinement to address reward hacking—suggesting that Agent RFT is best suited for high-value use cases where the engineering investment is justified by business impact.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
This case study explores OpenAI's approach to post-training and deploying large language models in production environments, featuring insights from a post-training researcher working on reasoning models. The discussion covers the operational complexities of reinforcement learning from human feedback at scale, the evolution from non-thinking to thinking models, and production challenges including model routing, context window optimization, token efficiency improvements, and interruptability features. Key developments include the shopping model release, improvements from GPT-4.1 to GPT-5.1, and the operational realities of managing complex RL training runs with multiple grading setups and infrastructure components that require constant monitoring and debugging.
Cosine, a company building enterprise coding agents, faced the challenge of deploying high-performance AI systems in highly constrained environments including on-premise and air-gapped deployments where large frontier models were not viable. They developed a multi-agent architecture using specialized orchestrator and worker models, leveraging model distillation, supervised fine-tuning, preference optimization, and reinforcement fine-tuning to create smaller models that could match or exceed the performance of much larger models. The result was a 31% performance increase on the SWE-bench Freelancer benchmark, 3X latency improvement, 60% reduction in GPU footprint, and 20% fewer errors in generated code, all while operating on as few as 4 H100 GPUs and maintaining full deployment flexibility across cloud, VPC, and on-premise environments.
