## Overview Brex, an all-in-one corporate spend and cash management platform, has successfully deployed a production AI assistant that revolutionizes expense management workflows for their enterprise customers. The company identified a significant opportunity in the financial services sector where manual processes like expense reporting, receipt management, and policy compliance create substantial friction and inefficiencies. Their solution, launched in early access in summer 2023 and generally available in January 2024, represents a mature LLMOps implementation that has processed millions of transactions and saved hundreds of thousands of hours for their customers. The core problem Brex addressed is endemic across corporate finance: employees waste significant time on repetitive tasks like writing expense memos, attaching receipts, and navigating complex compliance requirements. Finance teams spend countless hours reviewing transactions manually, often catching policy violations after the fact. This creates a cascade of inefficiencies where skilled professionals are bogged down in administrative work rather than strategic financial analysis. ## Technical Architecture and LLMOps Implementation Brex's technical approach centers around Amazon Bedrock as their foundational LLM platform, specifically leveraging Claude models for their natural language processing capabilities. This choice was strategic rather than arbitrary - Bedrock provided enterprise-grade security guarantees that keep customer financial data within Brex's AWS security boundary, a critical requirement for fintech applications. The integration allowed them to avoid infrastructure complexity while maintaining the security posture required for handling sensitive financial information. A key architectural component is their custom-built LLM Gateway, developed in March 2023, which serves as an intelligent routing and safety layer for all AI interactions. This gateway represents sophisticated LLMOps engineering, acting as more than just a simple proxy. It dynamically routes requests to appropriate models based on the complexity and nature of the task, standardizes responses across different model providers, implements comprehensive logging for monitoring and debugging, and provides centralized cost and rate limiting controls. The gateway's design demonstrates mature LLMOps thinking by abstracting the underlying model infrastructure from application code. Applications make standard API calls using OpenAI or Anthropic client libraries with overridden base URLs, allowing the gateway to transparently route requests to the most appropriate model or service. This architecture enables seamless model switching and A/B testing without requiring changes to downstream applications. ## Multi-Model Orchestration Strategy Brex employs a sophisticated multi-model orchestration approach that optimizes for both cost and performance. Rather than using a single powerful model for all tasks, they intelligently route different types of requests to appropriately sized models. Simple classification tasks or keyword extraction might be handled by lightweight, fast models, while complex policy interpretation or nuanced language generation is routed to more capable models like Claude. This orchestration extends beyond simple routing to include model chaining, where traditional machine learning outputs serve as inputs to LLMs for interpretation. For example, they use similarity search to identify documentation from similar past expenses, then provide this context to help inform how the LLM should approach new scenarios. This hybrid approach leverages the strengths of both traditional ML and generative AI, creating a more robust and cost-effective solution. The system also implements intelligent caching to avoid redundant API calls. When multiple users ask similar questions like "What's my travel budget?", the system reuses results for a specified period, significantly reducing both costs and latency. This type of optimization is crucial for production LLMOps deployments where cost management directly impacts business viability. ## Quality Assurance and Compliance Framework One of the most sophisticated aspects of Brex's LLMOps implementation is their dual-layer quality assurance system. The first layer involves AI-generated compliance information using various context clues including receipt data, calendar information, and transaction details to craft appropriate expense justifications. However, recognizing that different companies have varying standards for what constitutes adequate justification, they implemented a second layer: an AI compliance judge. This compliance judge, powered by another LLM, evaluates all expense submissions - whether generated by AI or humans - against company-specific standards for completeness, clarity, and correctness. Importantly, AI-generated content receives no special treatment; it undergoes the same rigorous evaluation as human-generated submissions. This approach demonstrates mature thinking about AI deployment in regulated industries, where maintaining consistent standards regardless of content source is crucial for audit and compliance purposes. The system is designed to err on the side of caution when quality is questionable, pushing back on users to provide additional clarity rather than accepting potentially non-compliant submissions. This conservative approach helps maintain the high compliance rates they've achieved while building trust with enterprise customers who are often skeptical of AI-generated content in financial contexts. ## Production Deployment and Scaling Considerations Brex's deployment strategy reveals several important LLMOps lessons learned through nearly two years of production operation. Initially, they launched the assistant as a chat interface, assuming that conversational AI would be the preferred user experience. However, they quickly discovered that most interactions fell into two categories: simple Q&A that could be answered inline, or complex requests requiring richer UI components. This insight led them to redesign the interface around their search functionality, providing Google-like inline AI responses while maintaining traditional search results below. This evolution demonstrates the importance of user experience considerations in LLMOps deployments - technical capability must be matched with appropriate interface design to achieve user adoption. The integration of external data sources proved crucial for improving LLM decision quality. By connecting calendar data, location information, and transaction details, the system can reason more contextually about expenses. For example, an Uber ride taken 30 minutes before a calendar event labeled "client dinner" can be reasonably classified as business-related. These data integrations require sophisticated pipeline management and represent a significant LLMOps engineering challenge in terms of data freshness, quality, and privacy. ## Monitoring, Evaluation, and Continuous Improvement Brex has implemented comprehensive monitoring and evaluation systems that track both technical performance metrics and business outcomes. They monitor usage patterns, model performance, cost metrics, and user satisfaction across their customer base. The ability to demonstrate concrete business value - 75% automation of expense workflows, hundreds of thousands of hours saved monthly, and compliance improvements from 70% to mid-90s - has been crucial for continued investment and customer adoption. Their approach to model evaluation and improvement includes systematic A/B testing of different models and approaches. As new models become available, whether new versions of Claude or Amazon's Titan models, they can systematically evaluate performance against their existing benchmarks and gradually migrate traffic to better-performing models through their gateway architecture. The system maintains detailed audit trails of all AI actions and decisions, which is crucial for regulated industries. This traceability allows them to investigate any issues that arise and provides the documentation needed for compliance audits. Users maintain control over AI functionality, with options to disable features or intercept requests for particularly cautious customers. ## Lessons Learned and Future Evolution Several key insights have emerged from Brex's production LLMOps experience. First, customer attitudes toward AI have evolved significantly over the deployment period. Initial hesitation and requests to disable AI features have transformed into AI capabilities being a competitive differentiator and reason for customers to choose Brex. This shift reflects broader market acceptance but also validates their careful approach to building trust through transparent, controllable AI systems. Second, customer expectations have evolved from simple automation of existing processes to demands for more comprehensive end-to-end automation. Early adopters initially valued AI for speeding up data entry, but now expect full workflow automation where AI handles the primary work while humans supervise. This evolution is pushing them toward what they describe as "inverting control" - moving from AI as a co-pilot to AI as the primary actor with human oversight. Third, the importance of data integration cannot be overstated. The most significant improvements in AI decision quality came not from better models or prompts, but from integrating rich contextual data from multiple systems. This insight has implications for how they approach future LLMOps projects and emphasizes the importance of comprehensive data architecture in AI success. Looking forward, Brex is expanding their AI capabilities to adjacent areas in finance such as accounting and back-office operations. They're also exploring multi-agent architectures to break their assistant into specialized agents that can collaborate on complex tasks. Integration with real-time data warehouses will enable more sophisticated financial analysis and recommendations. ## Business Impact and Validation The measurable impact of Brex's LLMOps implementation provides strong validation of their approach. Beyond the headline metrics of 75% automation and hundreds of thousands of hours saved, they've achieved significant improvements in compliance rates and user satisfaction. Some top customers report nearly 99% employee compliance, effectively eliminating friction between employees and finance teams. The cost savings extend beyond direct time savings to include reduced errors, faster month-end closes, and improved audit outcomes. Finance teams can focus on strategic activities like budget analysis and business guidance rather than chasing receipts and correcting errors. The improved user experience has transformed expense reporting from a punitive process to one where employees feel supported and guided. This comprehensive LLMOps implementation demonstrates how thoughtful application of large language models can transform traditional business processes when combined with proper architecture, quality controls, and user experience design. Brex's success provides a template for other organizations looking to implement production AI systems in regulated industries where trust, compliance, and reliability are paramount.