## Company and Use Case Overview PerformLine operates in the marketing compliance industry, providing comprehensive oversight across marketing, sales, and partner channels for consumer finance brands and global organizations. The company has conducted over 1.1 billion compliance observations over the past decade, automating compliance processes from pre-publication review to continuous monitoring of consumer-facing channels like websites, emails, and social media. Their core mission follows the principle of "Discover. Monitor. Act," transforming compliance efforts into competitive advantages for their clients. The specific challenge addressed in this case study involved one of PerformLine's enterprise customers who needed more efficient compliance checks on newly launched product pages. These pages presented particular complexity as they integrated multiple products within the same visual and textual framework, featuring overlapping content that could apply to one product, several products, or all products simultaneously. This required context-aware interpretation that mirrors how typical consumers would view and interact with the content, moving beyond traditional static parsing approaches. ## Technical Architecture and LLMOps Implementation PerformLine developed a sophisticated serverless, event-driven architecture built on AWS services that seamlessly integrated with their existing systems. The solution was remarkably efficient to implement, requiring less than a day to develop and deploy, allowing the team to focus on prompt optimization, evaluation, and cost management rather than infrastructure overhead. The architecture implements a comprehensive data flow starting with millions of pages processed by upstream ETL processes from PerformLine's core systems. When pages are retrieved, they trigger events in the compliance check system, with Amazon S3 storing page data according to metadata. Amazon EventBridge routes S3 events to Amazon SQS, which queues messages for processing and enables retry mechanisms on failure. The core processing occurs through AWS Lambda functions that consume SQS messages and scale dynamically to handle unpredictable workloads. These functions leverage Amazon Bedrock for extraction and generative AI analysis of content, with PerformLine strategically selecting different models based on specific requirements. Amazon Nova Pro was chosen for complex requests requiring powerful analysis while maintaining a high performance-to-cost ratio, while Anthropic's Claude Haiku was utilized for optimized quick calls where fast response times were paramount. ## Prompt Engineering and Model Management A critical aspect of PerformLine's LLMOps implementation involved sophisticated prompt engineering and management strategies. Initially, they faced challenges with manually tracking multiple prompt versions and templates, which became inefficient as they iterated and collaborated on improvements. This led them to adopt Amazon Bedrock's Prompt Management service, which provided centralized versioning, management, and seamless deployment of prompts to production. The prompt management system allows for dynamic referencing in AWS Lambda functions, enabling flexible configuration without code redeployment. PerformLine combined this with Amazon Bedrock application profile inference endpoints, allowing them to dynamically adjust models invoked by Lambda functions, track costs per invocation, and attribute expenses to specific application instances through cost tags. To streamline model interactions, they chose the Amazon Bedrock Converse API, which provides a standardized interface for model invocation. This combination of inference endpoints, prompt management, and the Converse API created a highly configurable system where developers could rapidly test new models and prompts, evaluate results, and iterate without rebuilding or redeploying applications. ## Cost Optimization Strategies PerformLine implemented several sophisticated cost optimization techniques that demonstrate mature LLMOps practices. Their change data capture (CDC) approach involves writing analyzed and formatted page content back to partitions that include metadata hashes of assets. This enables upstream processes to determine whether pages have already been processed and if content has changed, resulting in significant efficiency gains. The multi-pass inference strategy represents a particularly sophisticated approach to balancing cost and accuracy. Rather than processing all content with expensive, powerful models, PerformLine implemented a tiered approach using different Amazon Bedrock models based on processing requirements. Initial filtering occurs with Amazon Nova Micro, a lightweight model that efficiently identifies relevant products with minimal cost. Identified products are then batched into smaller groups and passed to Amazon Nova Lite for deeper analysis, keeping within token limits while improving extraction accuracy. This context-aware processing approach first identifies target content and then processes it in smaller batches, significantly improving accuracy while minimizing token consumption. The strategy demonstrates sophisticated understanding of how to optimize LLM inference costs at scale while maintaining the precision critical for compliance applications. ## Production Scale and Performance The production implementation demonstrates impressive scale, with PerformLine projected to process between 1.5 to 2 million pages daily. From this volume, they expect to extract approximately 400,000 to 500,000 products, with rules applied to each asset resulting in about 500,000 rule observations requiring daily review. This scale necessitated careful attention to performance optimization and cost management. The system achieved measurable business impact through intelligent automation. PerformLine experienced a 15% reduction in human evaluation workload, freeing time for human evaluators to focus on critical pages rather than processing all pages indiscriminately. Additionally, by avoiding reprocessing of unchanged pages, they achieved over 50% reduction in analyst workload beyond the deduplication gains, demonstrating the compound benefits of intelligent content management. ## Integration and Workflow Management The architecture demonstrates sophisticated integration capabilities with existing systems. Customer-defined product schemas are stored in Amazon DynamoDB, enabling dynamic large language model targeting and schema-driven output generation. The extracted data is formatted as structured JSON adhering to target schemas and stored in Amazon S3, with EventBridge forwarding S3 events to Amazon SQS to make extracted data available for downstream processing. This design ensures that compliance checks and business rules running on other PerformLine systems can seamlessly validate and enforce regulatory requirements without disrupting existing workflows. The event-driven nature of the architecture provides natural decoupling between components, enabling independent scaling and maintenance of different system elements. ## Operational Considerations and Future Enhancements PerformLine's implementation demonstrates mature thinking about operational aspects of LLMOps. The system includes multiple queue types (Incoming, DLQ, Results) with comprehensive error handling mechanisms. The serverless approach with Lambda functions provides automatic scaling capabilities essential for handling variable workloads in compliance monitoring scenarios. Looking toward future enhancements, PerformLine has identified several Amazon Bedrock features for exploration, including prompt caching and Amazon Bedrock Flows. Prompt caching offers potential for up to 85% latency improvements and 90% cost reduction compared to calls without caching, which PerformLine sees as becoming standard practice. This capability would enable further analysis on the same content at lower cost, creating new opportunities for feature expansion and development. Amazon Bedrock Flows represents a next step in simplifying orchestration of knowledge bases, prompt caching, and potentially Amazon Bedrock agents. The visual workflow builder could help reduce time to feature deployment and maintenance, further streamlining their LLMOps processes. ## Technical Assessment and Considerations While the case study presents impressive results, it's important to note that this is a co-authored post with AWS, which may emphasize positive outcomes and AWS service benefits. The claimed development time of "less than a day" for such a comprehensive solution seems optimistic and may not account for the full cycle of requirements gathering, testing, and production validation that typically accompanies enterprise deployments. The technical architecture itself appears sound, with appropriate use of event-driven design patterns and serverless technologies for variable workloads. The multi-pass inference strategy demonstrates sophisticated understanding of LLM cost optimization, though the specific cost savings figures should be validated against actual production metrics over extended periods. The integration approach of maintaining compatibility with existing downstream systems while introducing AI-powered processing represents a pragmatic approach to LLMOps adoption. However, the complexity of managing multiple model types, prompt versions, and inference profiles suggests ongoing operational overhead that may require dedicated expertise to maintain effectively. Overall, this case study represents a mature approach to implementing LLMs in production for compliance use cases, with thoughtful attention to cost optimization, scalability, and integration concerns that are characteristic of successful LLMOps implementations.