PayU is a Central Bank-regulated financial services provider in India that offers a full-stack digital financial services system serving merchants, banks, and consumers. The company faced a common enterprise challenge in the age of generative AI: employees were increasingly using public AI assistants for work tasks including technical troubleshooting, email generation, and content refinement, but this created significant security and compliance risks given their status as a regulated financial institution with strict data residency requirements. The core problem centered around the tension between employee productivity gains from AI tools and the company's obligation to protect sensitive data including proprietary system information, confidential customer details, and regulated documentation. As a Central Bank-regulated entity, PayU was required to keep all data within India and securely contained within their virtual private cloud (VPC), making the use of external AI services a non-starter from a compliance perspective. ## Technical Architecture and LLMOps Implementation PayU's solution represents a comprehensive enterprise LLMOps implementation built around Amazon Bedrock as the core foundation model service. The architecture demonstrates several key LLMOps patterns including secure model access, role-based permissions, RAG implementation, and agentic workflows. The frontend layer uses Open WebUI, an open-source, self-hosted application that provides a user-friendly interface for interacting with large language models. This choice reflects a pragmatic LLMOps decision to use proven open-source tooling while maintaining full control over the deployment environment. Open WebUI was containerized and deployed on Amazon EKS with automatic scaling, demonstrating production-ready orchestration practices. The system integrates with the company's identity provider for single sign-on and implements role-based access control (RBAC) tied directly to job functions, ensuring that teams only access AI capabilities relevant to their responsibilities. A custom Access Gateway serves as an intermediary between Open WebUI and Amazon Bedrock, translating Amazon Bedrock APIs to a schema compatible with the frontend. This architectural pattern is common in enterprise LLMOps where organizations need to integrate multiple AI services behind a unified interface while maintaining flexibility to swap out underlying models or services. ## Model Management and Multi-Agent Architecture The solution leverages Amazon Bedrock's diverse selection of foundation models from providers including AI21 Labs, Anthropic, Cohere, DeepSeek, Meta, Mistral AI, Stability AI, and Amazon. Rather than implementing a single monolithic AI assistant, PayU developed specialized agents tailored to specific business functions: `hr-policy-agent`, `credit-disbursal-agent`, `collections-agent`, and `payments-demographics-agent`. This multi-agent approach reflects mature LLMOps thinking about domain specialization and controlled access to different data sources. The agents employ different architectural patterns depending on their use cases. The HR policy agent uses a traditional RAG approach, querying vectorized knowledge bases stored in Amazon OpenSearch Service. In contrast, the credit disbursement agent implements a text-to-SQL pipeline that translates natural language queries into structured SQL commands to extract insights from their Amazon S3-based data lakehouse. This hybrid approach demonstrates sophisticated LLMOps design where different retrieval and generation patterns are matched to specific data types and query requirements. ## Data Infrastructure and RAG Implementation PayU's data infrastructure, internally called "Luna," is built using Apache Spark and Apache Hudi, with business-specific datamarts stored in Amazon S3 in highly denormalized form enriched with metadata. The data is exposed through AWS Glue tables functioning as a Hive Metastore and can be queried using Amazon Athena. This represents a well-architected data lakehouse approach that supports both traditional analytics and modern LLMOps requirements. For the RAG implementation, HR policy documents are stored in S3 buckets and vectorized using Amazon Bedrock Knowledge Bases, with vector embeddings stored in OpenSearch Service. The vectorization and semantic search capabilities enable employees to query policy information using natural language while ensuring responses are grounded in authoritative company documentation. ## Agent Orchestration and Workflow Management The text-to-SQL workflow demonstrates sophisticated agent orchestration using Amazon Bedrock Agents. The system includes instruction prompts that guide the orchestration process, with agents interpreting prompts and coordinating workflows by delegating specific actions to underlying LLMs. The orchestration includes error handling and query validation steps, where agents are instructed to check SQL syntax and fix queries by reading error messages before executing final queries. Action groups within Amazon Bedrock Agents organize and execute multiple coordinated actions in response to user requests. Each action group includes schemas that define required formats and parameters, enabling accurate interaction with the compute layer through AWS Lambda functions. The Lambda functions serve as execution engines, running SQL queries and connecting with Athena to process data, with appropriate resource policies and permissions configured for secure serverless operation. ## Security and Compliance Architecture Given the sensitive nature of financial data, PayU implemented AWS PrivateLink to create a private, dedicated connection between their VPC and Amazon Bedrock. This ensures that organizational data included as context in prompts and generated responses containing sensitive information never traverse the public internet. The PrivateLink implementation creates an interface endpoint that provisions a network interface directly in their VPC subnet, making Amazon Bedrock accessible as though it resides within their own VPC while eliminating the need for internet or NAT gateways. Amazon Bedrock Guardrails provides essential safeguards across model, prompt, and application levels, helping block undesirable and harmful content while filtering hallucinated responses in both RAG and agentic workflows. This multi-layered security approach addresses both technical security concerns and regulatory compliance requirements for financial services. ## Production Monitoring and Governance The system stores configurations, user conversation histories, and usage metrics in a persistent Amazon RDS PostgreSQL database, enabling audit readiness and supporting compliance requirements. This approach to conversation logging and usage tracking is essential for enterprise LLMOps, particularly in regulated industries where audit trails and usage monitoring are mandatory. Role-based access control extends beyond simple user permissions to encompass access to specific agents, knowledge bases, and foundation models based on job functions. This granular approach to access control reflects mature thinking about AI governance in enterprise environments where different roles require access to different types of information and capabilities. ## Results and Business Impact PayU reports a 30% improvement in business analyst team productivity following the rollout, though this figure should be interpreted with appropriate caution as it comes from internal estimates rather than rigorous measurement. The productivity gains are attributed to analysts being able to focus on more strategic tasks with reduced turnaround times. The solution has generated significant interest in generative AI across the organization and has led to collaboration between business units and technical teams, accelerating digital transformation efforts. ## Critical Assessment and LLMOps Lessons This case study demonstrates several important LLMOps patterns and considerations. The multi-agent architecture shows how enterprise AI systems can be designed with domain-specific capabilities rather than attempting to build a single general-purpose assistant. The hybrid approach combining RAG and text-to-SQL demonstrates how different retrieval and generation patterns can be matched to different data types and query requirements. The security-first approach with PrivateLink and comprehensive access controls reflects the reality that many enterprise LLMOps implementations must prioritize compliance and security over convenience or cost optimization. The choice to use open-source frontend tooling (Open WebUI) while leveraging managed AI services (Amazon Bedrock) represents a balanced approach to build-versus-buy decisions common in enterprise AI implementations. However, the case study is presented as an AWS customer success story, so claims about productivity improvements and user satisfaction should be evaluated with appropriate skepticism. The technical architecture appears sound and well-designed for the stated requirements, but the business impact metrics would benefit from more rigorous measurement and independent validation. The solution represents a mature approach to enterprise LLMOps that addresses real-world constraints around security, compliance, and organizational governance while still delivering meaningful AI capabilities to end users. The implementation patterns demonstrated here - multi-agent architectures, role-based access control, private model access, and hybrid RAG approaches - are likely to be relevant for other regulated industries facing similar challenges in deploying generative AI capabilities securely.