## Overview AWS Sales developed an internal generative AI-powered assistant to streamline the process of creating account plans (APs), which are comprehensive strategy documents that sales personnel draft for established AWS customers. Prior to this solution, account managers spent up to 40 hours per customer creating these documents, leading to significant organizational overhead. The solution, launched in October 2024, builds on an existing internal tool called Field Advisor and leverages Amazon Bedrock to help sales teams create comprehensive and insightful APs in less time. It's worth noting that this case study comes from AWS itself, describing an internal tool built using their own cloud services. While this provides interesting insights into how AWS uses its own products, readers should be aware that it also serves as a showcase for AWS services. The reported benefits and time savings come from internal testimonials rather than independent verification. ## Business Problem and Use Cases The account plans draft assistant addresses four primary use cases: The first is account plan draft generation, where Amazon Bedrock is used to make internal and external data sources available for generating draft content for key sections of APs. This enables sales teams to quickly create initial drafts for sections such as customer overviews, industry analysis, and business priorities, which previously required hours of research across the internet and disparate internal AWS tools. The second is data synthesis, where the assistant pulls relevant information from multiple sources including CRM systems, financial reports, news articles, and previous APs to provide a holistic view of customers. This multi-source aggregation is a classic use case for RAG-based systems. Third, built-in quality checks help ensure that APs meet internal standards for comprehensiveness, accuracy, and strategic alignment. This quality assurance capability is an important aspect of production LLM systems, though the case study doesn't elaborate on how these checks are implemented. Finally, the system offers customization capabilities, allowing account managers to upload proprietary documents to match their unique customer knowledge and strategic approach while still benefiting from AI-generated drafts. ## Technical Architecture The solution employs a sophisticated multi-component architecture built on AWS services. When a user initiates the workflow in the internal CRM system, it triggers the account plan draft assistant through a pre-signed URL. The assistant then orchestrates a multi-source data collection process, performing web searches while also pulling account metadata from OpenSearch, Amazon DynamoDB, and Amazon S3 storage. ### Core Components **Amazon Bedrock** serves as the foundation, providing programmatic API access to high-performing foundation models along with vector search capabilities and metadata filtering using Amazon Bedrock Knowledge Bases. The knowledge bases are populated using sales-enablement materials, historic APs, and other relevant documents curated by AWS Glue jobs. This RAG architecture enables the system to semantically search and use metadata filtering to retrieve relevant context from diverse sources. **AWS Lambda** functions support two distinct use cases in the architecture. The async resolver Lambda function interfaces with the front-end client CRM and orchestrates async job IDs for the client to poll. This layer also handles input validations, user request throttling, and cache management. Worker Lambda functions perform the actual content generation, working concurrently to generate different sections of APs by using publicly available data, internal data, and curated data in the Bedrock knowledge bases. These functions invoke various LLMs using Amazon Bedrock and store the final content in the AP's DynamoDB database corresponding to each async job ID. **Amazon DynamoDB** maintains the state of each user request by tracking async job IDs, tracks throttling quota (both global request count and per-user request count), and acts as a cache. This stateful tracking is essential for the asynchronous processing model the system employs. **AWS Glue jobs** curate and transform data from various internal and external data sources. These jobs push data to internal data sources (APs, internal tooling team S3 buckets, and other internal services) and to Bedrock Knowledge Bases, facilitating high-quality output through RAG. **Amazon SQS** enables decoupling between the management plane and data plane. This decoupling is crucial in allowing the data plane worker functions to concurrently process different sections of the APs and ensure that APs can be generated within specified timeframes. The system also includes a custom web frontend built using a ReactJS-based micro-frontend architecture, which integrates directly into the CRM system for a seamless user experience. This allows account managers to access the assistant without leaving their familiar work environment. ## Data Management and RAG Implementation The solution uses Amazon Bedrock's out-of-the-box knowledge base management capabilities. Through its RAG architecture, the system semantically searches and uses metadata filtering to retrieve relevant context from diverse sources including internal sales enablement materials, historic APs, SEC filings, news articles, executive engagements, and data from CRM systems. The connectors built into Amazon Bedrock handle data ingestion from Amazon S3, relational database management systems (RDBMS), and third-party APIs. The knowledge base capabilities enable filtering and prioritization of source documents when generating responses, resulting in what AWS describes as higher quality and more relevant content in generated AP sections. ## Security and Compliance Considerations The case study emphasizes that security and compliance are paramount when dealing with customer data. AWS IAM Identity Center is used for enterprise single sign-on so that only authorized users can access the account plans draft assistant. Through Field Advisor, various internal authorization mechanisms help ensure that users generating APs only access data they already have permissions for. This role-based access control is a critical aspect of any production LLM system handling sensitive business information. ## Operational Workflow The notification chain is another notable operational aspect. When content generation is complete, Amazon SQS and an internal notifications service API gateway deliver updates using Slack direct messaging and store searchable records in OpenSearch for future reference. This integration with existing communication tools (Slack) demonstrates thoughtful consideration of user workflow and adoption. ## Results and User Feedback Since its launch, thousands of sales teams have reportedly used the assistant to draft sections of their APs. According to testimonials included in the case study, one enterprise account manager reported saving at least 15 hours on their latest enterprise account plan, allowing more time for actual customer engagement rather than research and writing. A mid-market account manager noted that the tool helps rapidly generate baseline plans that can then be prioritized and customized, which is particularly valuable when managing multiple accounts. These testimonials should be taken with some caution as they are internal AWS employees discussing an internal AWS tool built with AWS services. However, the reported time savings from 40 hours down to significantly less represents a meaningful productivity improvement if accurate. ## Future Development Plans AWS outlines an ambitious roadmap toward creating what they call a "zero-touch account planner" that could generate a full AP for a customer, incorporating best practices observed across customers to provide sales teams with best-in-class strategies. Planned enhancements include deeper integration with purpose-built planning tools including automatic generation of value maps and stakeholder maps, enhanced personalization tailored by industry, account size, and individual user preferences, improved collaboration features for multiple team members to work together on refining AI-generated plans, and expanded recommendation capabilities to provide "what next?" ideas to sales teams. ## Key LLMOps Patterns Several notable LLMOps patterns emerge from this case study. The asynchronous processing model with job tracking addresses the latency challenges inherent in LLM-based content generation, allowing users to initiate requests and receive notifications when processing completes. The concurrent generation of different AP sections by separate worker Lambda functions demonstrates effective parallelization for complex multi-part document generation. The RAG architecture with metadata filtering shows a mature approach to grounding LLM outputs in relevant, authoritative sources. The throttling and quota management at both global and per-user levels indicates consideration for resource management and cost control in a production environment. The micro-frontend integration approach demonstrates how AI capabilities can be embedded into existing tools rather than requiring users to adopt entirely new workflows, which is often critical for enterprise adoption. Overall, this case study provides a detailed look at how a large technology organization has deployed LLM-based capabilities for a specific internal use case, with particular attention to the architectural decisions around asynchronous processing, data management, and security controls that are essential for production LLM systems.