Box, an enterprise content management platform, is building AI agents to bridge the gap between cutting-edge AI capabilities and enterprise knowledge work. The company faces the challenge of translating rapid progress in AI coding agents to broader business processes across marketing, sales, finance, and research. Box's solution involves creating BoxAgent, a specialized agent focused on working with enterprise content, built on top of their existing infrastructure for permissions, security, and governance. The platform leverages a coding-based harness approach, uses multiple models for different tasks, exposes capabilities both through native interfaces and headless APIs via MCP servers, and focuses on token efficiency and domain expertise in file systems and document processing. Early results show increased usage patterns where agents handle tasks that would never have been assigned to humans due to resource constraints, demonstrating the potential for 10-100x increases in data system utilization.
Box, a major enterprise content management platform, is actively deploying AI agents in production to help bridge the gap between frontier AI capabilities and practical enterprise knowledge work. The company’s CEO provides detailed insights into their LLMOps strategy, discussing both the technical architecture of their BoxAgent system and the broader organizational challenges of deploying AI in enterprise environments. This case study reveals a sophisticated approach to production AI that balances multiple models, maintains security and governance, and addresses the fundamental differences between deploying AI for coding versus general knowledge work.
Box identifies a critical gap in enterprise AI adoption: while AI coding agents have achieved remarkable success with developers experiencing significant productivity gains daily, translating these gains to other knowledge work domains like marketing, sales, finance, and research remains challenging. The company analyzes why coding agents work so well and identifies several favorable conditions that don’t exist elsewhere. Models are heavily trained on code due to abundant training data on the internet. The work itself is verifiable, allowing both reinforcement learning during training and validation during use. Users are highly technical and can troubleshoot issues, grant appropriate access to data sources, and make informed decisions about security when agents request access to external packages or CLIs. Developers typically have broad access to codebases with permissive permission structures.
In contrast, non-coding knowledge work faces significant obstacles. The work is less verifiable than code execution. Models are still developing proficiency in domain-specific non-coding tasks. Users lack technical expertise to handle complex agent configurations or troubleshoot failures. Most critically, enterprises have complex, granular permission structures where every individual and team has different data access patterns. Agents inherit these same permission limitations, creating a multiplicative complexity problem. When agents encounter access restrictions, they lack the social navigation capabilities humans use to request permissions from colleagues.
Box built BoxAgent as a specialized agent focused on enterprise content management, leveraging over a decade of architectural decisions that fortuitously benefit AI workloads. The platform benefits from fundamental design choices made years before AI agents became viable. Box maintains a single canonical way to store files, with only one ID system per object, one governance model, and one permissions architecture. This eliminates the ambiguity present in legacy competitors where data might exist in multiple forms or locations. The system provides unified access controls that agents can reliably navigate.
The document processing pipeline represents significant pre-AI infrastructure that proves essential for agent work. Box converts all content types into agent-friendly formats, with a dedicated API that transforms any content into markdown. This conversion pipeline runs at scale before documents ever reach an agent. The search infrastructure required modification for agent use cases. Box discovered that agents can consume 100x more search results than humans, leading them to reconsider ranking algorithms. They now prioritize providing context within search results over pure ranking, since agents can perform their own ranking and synthesis across larger result sets.
Box takes a deliberately multi-model approach in production. Different models excel at different tasks: one might be superior for coding, another for legal contracts, a third for insurance claims. The team uses evaluations to map capabilities to models. Performance characteristics vary significantly, with some models like Gemini Flash offering faster inference than models like Opus 4, making them suitable for different workload profiles. Cost optimization becomes increasingly important as enterprise adoption scales, with the potential for significant savings by routing tasks to appropriately-sized models rather than always using the most expensive frontier models. However, Box notes practical constraints around prompt caching that create penalties for switching between models, requiring careful optimization of when to use hybrid approaches versus committing to a single model for a session.
The agent harness itself is built on coding primitives, reflecting Box’s philosophy that agents will always be better at code than other tasks, so this capability should be leveraged even for non-coding work. This approach essentially provides unlimited engineering resources to every knowledge worker. A marketing manager or sales analyst can have agents write code on-the-fly to automate campaigns or extract customer intelligence from multiple systems, tasks that would have required dedicated engineering resources that were never available. The harness includes computer access, sandboxed environments for code execution, and connectors to external systems.
Box invests heavily in domain-specific customization within this coding harness framework. The system prompt and various harness components encode expertise about file systems, teaching the agent when to use search tools versus navigating folder paths. This represents a sophisticated understanding that search doesn’t always provide surrounding context, while folder navigation reveals the workspace structure and related documents. The agent learns to assess task types and select appropriate navigation strategies, demonstrating the kind of nuanced decision-making that separates domain-specific agents from general-purpose systems.
Box pursues a dual strategy for agent deployment, acknowledging that customer preferences will ultimately determine adoption patterns. The native BoxAgent interface focuses on scenarios where working with content is paramount, where token efficiency matters most, and where model selection requires domain expertise. Box builds specialized agents like their document extraction agent, which they believe can outperform any horizontal system due to their focused optimization, custom evaluations, and singular problem focus.
Simultaneously, Box embraces headless deployment through MCP servers, recognizing that many users will interact with Box content through other agent interfaces like Claude or ChatGPT. The company philosophically commits to being “completely indifferent” to which approach customers choose, though they recognize that by volume, most usage will likely be headless. The Salesforce announcement of fully exposing APIs through MCP represented a symbolic moment for the industry, with the world’s largest SaaS company validating the headless approach.
Box has already observed the practical implications of headless access through their own use of MCP servers from other vendors. The CEO describes using Salesforce’s MCP server to conduct market research tasks that would take 20 minutes of agent work equivalent to hours of human work, but which would never have been assigned to a person because the value didn’t justify the human time commitment. This illustrates the Jevons paradox: when a resource becomes cheaper and more accessible, total consumption increases dramatically. Agents enable entirely new categories of tasks that create value but fall below the threshold that would justify human effort.
Token cost management emerges as a central concern for production LLMOps at enterprise scale. Box observes that the “token-maxing meme” lasted approximately three days in practical reality, as only extremely well-capitalized companies can sustain unlimited token usage. While venture-funded startups can convert funding into token consumption, public companies and traditional enterprises face earnings-per-share targets and quarterly financial pressures that prevent sudden multi-million-dollar increases in AI bills.
This economic reality drives several architectural decisions. Box uses their product roadmap as a control mechanism for token utilization, encouraging productive use while avoiding perverse incentives that would reward token consumption for its own sake. The team pushes to stack more features and capabilities into the roadmap, with the understanding that valuable work will naturally drive appropriate token usage.
The company anticipates a significant shift in the industry as cost pressures mount. They predict a threshold where cost savings from task-specific optimization justify dedicated harnesses for high-value workflows. For example, an insurance claims process that could generate or save 50 million dollars per year justifies hyper-tuning to specific models, capabilities, and custom instructions. Not every workflow meets this threshold—simple document summarization doesn’t warrant such investment—but heavy-duty enterprise workflows increasingly do. As frontier intelligence remains expensive while frontier-minus-one intelligence from three months prior becomes substantially cheaper, Box expects enterprises to peel off workloads to more cost-effective models, creating demand for harnesses that can handle model neutrality and intelligent routing.
Box’s existing infrastructure for security and governance provides significant advantages for agent deployment. The single permissions architecture means agents inherit clear, well-defined access controls without ambiguity. The governance model that applies to human users applies equally to agents, providing consistent policy enforcement. Document conversion and processing happens in secure, governed pipelines that maintain access controls throughout transformation processes.
However, Box also recognizes new challenges introduced by agents. The permission problem multiplies in complexity because each agent effectively inherits the permission profile of its associated user or workflow, and enterprises may deploy numerous agents with varying access patterns. The platform must handle scenarios where agents encounter permission boundaries gracefully, unlike humans who can socially negotiate access by asking colleagues for permissions.
Box maintains custom evaluations for their specialized agents, particularly around document processing tasks. These evaluations allow them to validate claims about outperforming horizontal systems and to make informed decisions about model selection. The evaluation framework helps identify which models perform best for specific document types or extraction tasks, feeding into their multi-model routing strategy.
The company’s decade of experience with enterprise content creates an evaluation advantage. They understand the variety of document formats, the edge cases in file system navigation, and the patterns of how enterprises structure content. This domain knowledge translates into more sophisticated evaluation scenarios that capture real-world complexity rather than synthetic benchmarks.
Box discusses emerging organizational patterns for enterprise AI adoption. Domain-specific and vertical-specific agent startups have an advantage because they can pre-configure MCP connectors, provide forward-deployed engineers who understand the vertical, and dramatically reduce the burden on end users to configure complex agent systems. For enterprises building internal agents, Box sees value in creating dedicated AI agent engineering roles—essentially internal forward-deployed engineers who understand both the technology and the organization’s specific workflows and data landscape.
The company emphasizes that the industry faces years of diffusion, deployment, change management, and adoption work. There will be sustained cognitive dissonance between the hyperbolic progress of AI coding agents and the slower advancement in other knowledge work domains due to the structural challenges discussed. This long diffusion timeline also informs Box’s perspective on AI safety concerns, as the practical rate of deployment into organizations creates natural brakes on takeoff scenarios.
Box’s experience reveals several important lessons for LLMOps practitioners. The architectural decisions made years before AI agents became viable proved surprisingly consequential. Single sources of truth, canonical ID systems, and unified governance models that seemed like engineering hygiene now provide competitive advantages. Infrastructure investments in document conversion, search engines, and permissions systems compound in value when agents enter the picture.
The company demonstrates that successful enterprise LLMOps requires balancing multiple competing concerns: native experiences versus headless APIs, single-model optimization versus multi-model flexibility, token maximization versus cost management, and general-purpose capabilities versus domain-specific expertise. Rather than making binary choices, Box actively pursues both sides of these tradeoffs, letting customer needs and specific use cases determine the appropriate approach.
The emphasis on coding-based harnesses even for non-coding work represents a significant architectural philosophy. By treating agents-that-can-code as the fundamental primitive, Box gains flexibility to generate custom automation for any knowledge work domain. This approach acknowledges that code remains the most reliable capability of current AI systems and leverages that strength rather than waiting for models to become equally proficient at all task types.
Finally, Box’s realistic assessment of adoption timelines and cost pressures provides a valuable counterweight to more optimistic narratives. The company operates in the practical reality of enterprise budgets, quarterly targets, and organizational change management, providing grounded insights into how AI deployment actually proceeds at scale rather than in well-funded experimental contexts.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Notion, a knowledge work platform serving enterprise customers, spent multiple years (2022-2026) iterating through four to five complete rebuilds of their agent infrastructure before shipping Custom Agents to production. The core problem was enabling users to automate complex workflows across their workspaces while maintaining enterprise-grade reliability, security, and cost efficiency. Their solution involved building a sophisticated agent harness with progressive tool disclosure, SQL-like database abstractions, markdown-based interfaces optimized for LLM consumption, and a comprehensive evaluation framework. The result was a production system handling over 100 tools, serving majority-agent traffic for search, and enabling workflows like automated bug triaging, email processing, and meeting notes capture that fundamentally changed how their company and customers operate.
Notion AI, serving over 100 million users with multiple AI features including meeting notes, enterprise search, and deep research tools, demonstrates how rigorous evaluation and observability practices are essential for scaling AI product development. The company uses Brain Trust as their evaluation platform to manage the complexity of supporting multilingual workspaces, rapid model switching, and maintaining product polish while building at the speed of AI industry innovation. Their approach emphasizes that 90% of AI development time should be spent on evaluation and observability rather than prompting, with specialized data specialists creating targeted datasets and custom LLM-as-a-judge scoring functions to ensure consistent quality across their diverse AI product suite.
