Jefferies, a global investment banking firm, built an agentic AI trade assistant to address the challenge of equities traders needing real-time insights from vast datasets without coding ability or IT dependencies. The solution uses Strands Agents SDK, Amazon Bedrock with Anthropic Claude, Amazon Bedrock Knowledge Bases, and Model Context Protocol (MCP) tools to enable traders to query millions of rows of trading data through natural language, generating SQL queries and dynamic visualizations in real-time. Since launch, the solution has delivered measurable efficiency gains across global sales and trading operations, democratized data access, reduced IT burden from manual dashboard creation, and allowed traders to redirect time toward client relationships and strategic decision-making rather than manual data analysis.
Jefferies, a global full-service investment banking firm, developed an agentic AI trade assistant to transform how its front office equities trading desks operate. The core problem was that traders needed real-time insights into client behavior, trade patterns, and market trends from millions of rows of data spread across multiple visualization tools, but lacked the time and coding ability to build and maintain the necessary analytical systems. Traditional approaches forced traders to rely on subject matter experts for analysis and collaborate with IT teams to build custom dashboards, a process that could take days or weeks, creating a significant gap between available data and actionable decisions.
The solution represents a production deployment of LLMs in a high-stakes financial trading environment where accuracy, speed, and security are paramount. This case study demonstrates enterprise-scale LLMOps practices including agent orchestration, retrieval-augmented generation (RAG), security guardrails, compliance controls, and integration with existing trading infrastructure.
The solution is built on a sophisticated multi-layered architecture that exemplifies modern LLMOps practices for production AI systems. At its foundation, the system uses Strands Agents, an open-source agent harness SDK that takes a model-driven approach to building and running AI agents. Strands simplifies agent development by embracing the advanced capabilities of LLMs to plan, chain thoughts, call tools, and reflect on their actions. This framework scales from straightforward to complex agent use cases and from local development to production deployment.
The core LLM infrastructure leverages Amazon Bedrock to provide access to Anthropic Claude Sonnet, which handles natural language understanding, SQL generation, and multi-step tool orchestration. The team chose Amazon Bedrock specifically for its flexibility to switch between different LLMs as the trade assistant evolves, demonstrating forward-thinking LLMOps practices around model flexibility and avoiding vendor lock-in at the model level.
Amazon Bedrock Knowledge Bases provides a managed RAG pipeline that stores embedded representations of the underlying data model metadata, including table schemas, column definitions, and query patterns. The system uses Amazon Titan Embeddings for semantic retrieval to surface relevant database schemas and query patterns. When a trader poses a natural language question, the Query Agent first queries the Knowledge Base to retrieve relevant schema context, which is then used to inform accurate SQL construction. This RAG approach is critical for grounding the LLM’s outputs in the actual data structure and reducing hallucinations in SQL generation.
The architecture implements Model Context Protocol (MCP) tools, where each data source is exposed as a distinct tool that Strands Agents can invoke. This design provides three key advantages for production LLMOps: extensibility (new data sources can be added as additional tools without restructuring core architecture), separation of concerns (logic for interacting with each system is encapsulated within its own tool), and flexibility (the agent can dynamically select which tools to use based on each query). The system can query multiple data sources including in-memory databases, SQL databases, and Financial Information Exchange (FIX) message files.
The production workflow demonstrates sophisticated integration with existing enterprise infrastructure. The solution is deployed using Amazon Elastic Kubernetes Service (Amazon EKS) for the authentication service, which verifies that only authorized traders can access sensitive trading data. The system integrates with Jefferies’ on-premises Business Intelligence system called Global Flow Monitor (GFM) through an embedded AI assistant widget in the trading interface.
When a trader submits a query, the request flows through an eight-step orchestrated workflow: authentication, session management through a Bot Service that maintains context across multiple queries, intelligent query planning by the Strands Agent, LLM invocation through Amazon Bedrock, RAG retrieval from Knowledge Bases, SQL generation, query execution with row-level security filtering, and finally visualization rendering using a markdown UI library. The entire system maintains conversation context throughout the session lifetime, enabling traders to drill down on topics and explore data insights conversationally.
The solution implements multiple layers of security and compliance controls essential for production LLMOps in financial services. Amazon Bedrock Guardrails provide content moderation and personally identifiable information (PII) filtering to align with Jefferies’ policies. The system implements row-level data entitlements through a Query Executor that intercepts user requests and injects SQL filters to provide row-level security for data access, preventing accidental access to customer-sensitive data through intelligent access controls.
All conversations are logged for audit trails to meet regulatory compliance requirements in financial services. The authentication layer ensures that only authorized traders can access the system, and the row-level security ensures that traders can only access data they are entitled to see based on their role and permissions.
The Jefferies team implemented a deliberate hybrid approach to mitigate the risk of hallucinations and maintain data accuracy. Rather than relying on LLMs to generate visualizations directly, they separated concerns: the LLM handles natural language understanding and query generation, while dedicated visualization engines render the actual charts and graphs. This architectural decision preserves data accuracy while maintaining the conversational interface traders expect. The LLM selects which type of visualization to display (pie charts, bar charts, time series, etc.), but the actual rendering is done by trusted visualization libraries rather than LLM-generated code.
To meet the stringent performance requirements of real-time trading, the team made critical architectural decisions around data storage and processing. They use in-memory databases to maximize response times, as traders require split-second insights into client behavior, trade patterns, and market trends. Without this architectural choice, queries against the underlying dataset would have introduced unacceptable latency for real-time trading decisions. This demonstrates the importance of understanding domain-specific performance requirements in production LLMOps deployments.
The team adopted a deliberate multi-language strategy optimized for different aspects of the system. They rely on Python for LLM interactions and rapid experimentation, capitalizing on its rich AI/ML ecosystem and the extensive libraries available for working with LLMs and agentic frameworks. They implemented complex business processing in Java to leverage its performance characteristics for high-throughput data processing and integration with existing trading systems written in Java.
The team learned that real-world usage patterns proved dynamic, with traders interacting with the system in unexpected ways and their patterns shifting over time. They invested heavily in observability and user feedback loops to adapt quickly to these changing behaviors. This emphasis on monitoring and continuous improvement is a critical LLMOps practice for maintaining production AI systems, especially in domains where user behavior evolves as users discover new capabilities.
The Jefferies team identified several key lessons that serve as valuable guidance for enterprise LLMOps implementations. First, they validated the importance of not relying solely on LLMs for tasks where accuracy is paramount, instead using hybrid approaches that combine LLM reasoning with deterministic systems. Second, they learned that architectural decisions around data storage (in-memory databases) can be as critical as model selection for meeting domain requirements. Third, they discovered that user behavior evolves significantly once AI capabilities are deployed, requiring investment in feedback mechanisms and observability. Fourth, they validated the effectiveness of using the right language for each component (Python for AI/ML, Java for business logic and performance-critical processing).
Since launch, the trade assistant has delivered measurable efficiency gains across global sales and trading operations. Traders can now redirect time toward client relationships and strategic decision-making rather than manual data wrangling. The solution has materially reduced IT time previously consumed by repetitive dashboard creation, freeing technology teams for higher-value strategic initiatives. The ability to securely query millions of rows of equities trading data using natural language has democratized data access, fostering a data-driven culture where decisions are informed by real-time analytics rather than intuition or delayed reports.
The solution delivers personalized experiences by adapting to each trader’s query using MCP tools to access structured, unstructured, and in-memory data sources and dynamically generating graphs and charts in response. These efficiency gains translate directly into competitive advantage, as trading desks can now dedicate more capacity to onboarding clients and deepening existing relationships.
Jefferies’ roadmap signals continued investment in scaling this agentic AI approach. Plans include rolling out the Trade Assistant globally for multiple product types and desks beyond equities, enhancing audit capabilities with code generation tools that use natural language processing, and adding Amazon Bedrock AgentCore features to their enterprise AI system. This commitment to expansion demonstrates confidence in the production viability of the solution.
While this case study presents an impressive production deployment of agentic AI in financial services, several considerations warrant balanced assessment. The post is clearly promotional content for AWS services, and while it provides technical details, it does not discuss challenges, failures, or iterations that likely occurred during development. The claimed “measurable efficiency gains” are not quantified with specific metrics (e.g., percentage reduction in time to insights, number of IT hours saved, revenue impact), making it difficult to assess the true magnitude of business impact.
The architecture’s reliance on multiple AWS-specific services (Amazon Bedrock, Amazon Bedrock Knowledge Bases, Amazon EKS, Amazon Titan Embeddings) creates potential vendor lock-in, though the use of open standards like MCP and the open-source Strands Agents framework provides some portability. The post does not discuss model accuracy metrics for SQL generation, the error rates traders experience, or how the system handles ambiguous queries or edge cases. The claim that traders can query “millions of rows” through natural language is impressive but lacks detail on query complexity limits, failure modes, or how the system handles queries that span multiple data sources with conflicting information.
Nevertheless, this represents a genuine production deployment of LLMOps in a domain where accuracy, security, and compliance are critical. The architectural choices around RAG, MCP tools, security guardrails, and the hybrid approach to visualization generation demonstrate sophisticated thinking about production AI systems. The integration with existing trading infrastructure and the focus on maintaining conversation context shows practical understanding of user needs in high-pressure trading environments.
Stripe, processing approximately 1.3% of global GDP, has evolved from traditional ML-based fraud detection to deploying transformer-based foundation models for payments that process every transaction in under 100ms. The company built a domain-specific foundation model treating charges as tokens and behavior sequences as context windows, ingesting tens of billions of transactions to power fraud detection, improving card-testing detection from 59% to 97% accuracy for large merchants. Stripe also launched the Agentic Commerce Protocol (ACP) jointly with OpenAI to standardize how agents discover and purchase from merchant catalogs, complemented by internal AI adoption reaching 8,500 employees daily using LLM tools, with 65-70% of engineers using AI coding assistants and achieving significant productivity gains like reducing payment method integrations from 2 months to 2 weeks.
Octus, a leading provider of credit market data and analytics, migrated their flagship generative AI product Credit AI from a multi-cloud architecture (OpenAI on Azure and other services on AWS) to a unified AWS architecture using Amazon Bedrock. The migration addressed challenges in scalability, cost, latency, and operational complexity associated with running a production RAG application across multiple clouds. By leveraging Amazon Bedrock's managed services for embeddings, knowledge bases, and LLM inference, along with supporting AWS services like Lambda, S3, OpenSearch, and Textract, Octus achieved a 78% reduction in infrastructure costs, 87% decrease in cost per question, improved document sync times from hours to minutes, and better development velocity while maintaining SOC2 compliance and serving thousands of concurrent users across financial services clients.
Yahoo! Finance built a production-scale financial question answering system using multi-agent architecture to address the information asymmetry between retail and institutional investors. The system leverages Amazon Bedrock Agent Core and employs a supervisor-subagent pattern where specialized agents handle structured data (stock prices, financials), unstructured data (SEC filings, news), and various APIs. The solution processes heterogeneous financial data from multiple sources, handles temporal complexities of fiscal years, and maintains context across sessions. Through a hybrid evaluation approach combining human and AI judges, the system achieves strong accuracy and coverage metrics while processing queries in 5-50 seconds at costs of 2-5 cents per query, demonstrating production viability at scale with support for 100+ concurrent users.
