## Overview Parameta Solutions is a data provider within TP ICAP focused on over-the-counter (OTC) data solutions and advanced analytics for financial industry professionals. Their services span price discovery, risk management, and pre- to post-trade analytics. Like many financial services organizations, Parameta faced a scaling challenge with their client support operations: managing an increasing volume of email-based technical support requests efficiently while maintaining accuracy and response quality. The traditional manual process for handling client emails involved multiple steps: reading and understanding emails, extracting technical details, gathering relevant data from internal systems, determining proper routing, and verifying information against databases. This labor-intensive approach consumed significant time and introduced risks of human error that could impact client trust. The problem was exacerbated by inconsistent email structures and varying terminology between clients. Parameta sought a solution that could automate this workflow while maintaining their high standards of service, ultimately implementing an intelligent email triage system using Amazon Bedrock Flows. ## Technical Architecture The solution architecture comprises three main components: orchestration, structured data extraction, and intelligent response generation. ### Orchestration Layer Amazon Bedrock Flows serves as the central orchestrator for the entire email processing pipeline. The workflow begins when a client email arrives through Microsoft Teams, triggering a sequence that involves Amazon API Gateway to receive the request, an AWS Lambda function to extract email text, and Amazon S3 for storage. From there, Amazon Bedrock Flows coordinates the sequence of operations, managing conditional logic for different processing paths and enabling version management for controlled testing of prompt variations. This orchestration approach was chosen over alternatives like traditional NLP pipelines or separate ML classification models. Traditional NLP struggled with email complexity due to rigid rules and poor handling of language variations, while ML classification approaches would require separate specialized models for classification, entity extraction, and response generation, each with its own training data requirements. ### Structured Data Extraction The core of the email processing happens through a sequence of specialized prompts within the flow: - A classification prompt identifies the type of technical inquiry (e.g., price verification request) - An entity extraction prompt discovers key data points including product types, tickers, dates, and request types - A validation prompt verifies the completeness of required information for the identified query type These prompts work together to transform unstructured emails into actionable structured data. The case study provides a concrete example: a client email requesting verification of a closing price for a specific financial instrument is classified as a "price_verification_request" and has entities extracted including product_type, ticker, date_requested, data_source, and request_type in JSON format. ### Intelligent Response Generation The final stage leverages Amazon Bedrock Agents to synthesize information from multiple sources: - A knowledge base for technical documentation indexed in Amazon OpenSearch Service - Enterprise data accessed through Snowflake via Amazon Athena The agent queries both the knowledge base for product specifications and market context, and executes SQL queries against the data warehouse to retrieve specific pricing information. Response generation adapts based on validation results—providing specific information requests for incomplete queries or comprehensive solutions for complete inquiries. The generated response is then delivered back to clients through Microsoft Teams. ## LLMOps Considerations ### Deterministic LLM Workflows A key architectural decision was using Amazon Bedrock Flows to create deterministic, structured workflows rather than relying purely on raw LLM capabilities. This approach provided several operational advantages: - **Orchestrated prompt chaining**: Multiple specialized prompts work in a deterministic sequence, each optimized for specific tasks. This modular approach improves maintainability compared to monolithic prompts. - **Multi-conditional workflows**: Support for complex business logic with the ability to branch flows based on validation results or extracted information completeness. - **Version management**: The ability to switch between different prompt versions while maintaining workflow integrity enables rapid iteration without disrupting production pipelines. ### Model Selection Strategy The case study recommends diversifying model selection within the flow based on task complexity: - Using lighter, less expensive models for simple classification tasks - Reserving more advanced (and costly) models for complex reasoning - Creating resilience through model redundancy This approach optimizes both cost and performance by matching model capability to task requirements. ### Prompt Design Best Practices The case study outlines several prompt engineering principles that emerged from the implementation: - Design modular prompts that handle specific tasks for better maintainability - Keep prompts focused and concise to optimize token usage - Include clear input and output specifications for robustness - Include error handling considerations in prompt design - Create comprehensive test cases covering diverse scenarios ### Observability and Governance The structured nature of Amazon Bedrock Flows provides significant advantages for enterprise governance: - Comprehensive observability with holistic views of the end-to-end process - Built-in controls for appropriate oversight of the automated system - Transparent workflows that enable stakeholders to monitor, audit, and refine the system - Clear visibility into why emails were classified into specific categories, enhancing trust in automated responses - Ability to trace decisions through the workflow ### Version Control and CI/CD The case study emphasizes implementing proper CI/CD pipelines for flow deployment, establishing approval workflows for flow changes, and documenting flow changes along with their impact metrics. This reflects mature LLMOps practices where prompt and workflow changes are treated with the same rigor as traditional code deployments. ### Testing Approach Recommendations include creating comprehensive test cases covering diverse scenarios, validating flow behavior with sample datasets, and constantly monitoring flow performance and token usage in production. Starting with smaller workflows and scaling gradually is advised as a risk mitigation strategy. ### Cost Optimization The case study recommends regularly reviewing and optimizing prompt lengths, monitoring token usage patterns, and balancing model capability against cost when selecting models for different tasks within the flow. ## Results and Assessment According to the case study, the solution reduced resolution times from weeks to days and was developed in approximately two weeks. The low-code nature of Amazon Bedrock Flows reportedly enabled: - Development teams to accelerate prompt optimization through rapid testing - Support teams to understand and adjust the response process without deep AWS knowledge - Cross-functional collaboration on prompt improvements using familiar interfaces It's worth noting that this case study is published on the AWS blog and features testimonials from Parameta executives praising Amazon Bedrock Flows specifically. While the technical architecture and approach appear sound, readers should consider that the source may present an optimistic view of the implementation. The claims of reducing resolution from "weeks to days" are significant but lack specific metrics or methodology for measurement. Similarly, the two-week development timeline is impressive but may not account for all preparation, testing, and refinement work. The approach of using deterministic workflows with specialized prompts for different tasks (classification, extraction, validation) is a sensible architecture for production LLM systems, providing more control and observability than purely agentic approaches. The integration with enterprise data sources (Snowflake, Athena) and knowledge bases for RAG capabilities represents a realistic enterprise integration pattern. Overall, this case study demonstrates a practical application of LLMs in production for a relatively well-scoped problem domain (email triage), with appropriate attention to governance, observability, and operational concerns that are essential for enterprise deployments.