BlackRock, managing over $13 trillion in assets, developed a comprehensive responsible AI product to ensure trustworthy deployment of AI systems across highly regulated capital markets operations. The problem they faced was that AI hallucinations in financial services could lead to severe regulatory consequences, as illustrated by a near-miss incident where their agentic AI platform almost outputted a fabricated regulatory rule to portfolio managers handling billions in retirement funds. Their solution was to build responsible AI not as a feature or layer, but as a standalone product with eight mandatory real-time guardrails aligned to NIST standards, comprehensive offline and online evaluation pipelines, and deep integration with legal and compliance teams. The result was a reduction in compliance approval time from eight weeks to two weeks while ensuring no regulatory incidents, creating a trust system that institutional clients could rely on.
BlackRock’s case study presents one of the most comprehensive examples of responsible AI deployment in highly regulated industries. As one of the world’s largest asset managers with over $13 trillion in assets under management and 37 years of proprietary data, BlackRock operates in an environment where AI failures could trigger SEC investigations and severely damage institutional client trust. The Senior Vice President of AI Product shared insights into building both a responsible AI product and an agentic AI platform for the firm, with particular emphasis on the former.
The motivation for this rigorous approach stemmed from real-world incidents, including a May 2024 case where a major airline’s AI assistant hallucinated a bereavement refund policy that didn’t exist, leading to a tribunal case. While that incident involved a small refund, BlackRock recognized that a similar hallucination in their context—such as fabricating regulatory recommendations for portfolio managers overseeing multi-billion dollar pension portfolios—could result in SEC enforcement actions, making the stakes exponentially higher.
A pivotal moment occurred during the development of BlackRock’s agentic AI platform. In an initial prototype version being tested with a limited number of internal users, the system had only two basic guardrails implemented—what the team considered “table stakes” to satisfy compliance and legal requirements. When they expanded the user base, the model outputted a regulatory rule that did not exist. Critically, they caught this hallucination by accident. The existing checks that flagged the anomaly were not designed to catch fabricated citations or recommendations; the issue was caught as a side effect during manual review of another anomaly type.
This near-miss fundamentally changed the product roadmap and philosophy. The speaker candidly admitted to the common lie that AI product teams tell themselves: “I’ll build the core functionality first—the agentic platform, the tools registry, the model, the UI, the integrations—and I’ll add the guardrails later once users are engaged and executives are confident.” The reality check came when they realized they were one accident away from a portfolio manager receiving fabricated regulatory guidance that could have reached client memos, board meetings, and ultimately led to decisions affecting other people’s retirement money.
Following this incident, BlackRock adopted a firm-wide stance that represents a significant departure from typical AI development approaches: responsible AI cannot be a checkpoint, feature, or horizontal layer—it must be the product itself. This means the LLM models, user interfaces, prompts, and other components are treated as connective tissues to the central responsible AI product, rather than the responsible AI features being add-ons to application products.
The rationale for this inversion is rooted in the nature of what BlackRock is actually shipping to users. In highly regulated capital markets where they must answer to SEC, OCC, FINRA, and other regulatory bodies, they are essentially shipping a trust system that happens to have AI in it. Their institutional clients are described as stubborn and still skeptical about AI, so BlackRock is delivering a promise of trust with AI at the center, rather than an AI product with trust features bolted on.
This responsible AI product is mandatory across the firm—any AI output, prompt, application, or any AI input and output must plug into this offering as part of company policy. This architectural decision has profound implications for how AI systems are developed, deployed, and maintained at BlackRock.
The heart of BlackRock’s responsible AI product is a suite of eight real-time input and output moderation guardrails, aligned with the NIST (National Institute of Standards and Technologies) framework. NIST is a highly reputed standard widely used in capital markets, and this alignment ensures that the guardrails are not arbitrary thresholds but AI risk guardrails that are mandatory for capital markets applications. The eight guardrails span the spectrum from table stakes to highly specialized financial services requirements:
Table Stakes Guardrails: The toxicity guardrail prevents harmful outputs like instructions for creating weapons. The jailbreak and prompt injection guardrail blocks attempts to override system prompts or make the system behave inappropriately. The bias guardrail addresses discrimination based on religion, caste, creed, and political affiliation. The off-topic guardrail ensures the system doesn’t respond to queries outside its domain, such as directions to pharmacies or hospitals. The PII data privacy guardrail protects against exposing personally identifiable information based on an extensive list of items that could be considered PII.
Specialized Financial Services Guardrails: The investment advice guardrail is particularly noteworthy and represents significant innovation. BlackRock believes they may be the only firm in financial services with such a guardrail and are pursuing patents for it. This guardrail emerged from close collaboration with the legal team—a relationship the speaker characterized as initially difficult, given the typical product manager and engineer’s aversion to working with lawyers. The development process involved going through SEC rulings and FINRA rulings to create a proprietary guardrail for the US jurisdiction that can distinguish investment advice from other forms of financial information. The challenge was that regulatory rulings and guidelines don’t explicitly define what constitutes investment advice versus what doesn’t, requiring extensive legal interpretation work.
The hallucination and fabrication guardrail is directly motivated by the critical incident described earlier, designed to catch fabricated citations, regulatory recommendations, and other made-up information. The BlackRock sensitive content guardrail was developed in collaboration with the global affairs and public policy team to handle questions about the firm’s involvement in various matters, given that BlackRock’s products are public-facing and the company keeps a low profile, leading to creative and sometimes conspiratorial queries.
Beyond real-time guardrails, BlackRock has implemented comprehensive evaluation pipelines that span the pre-production to production lifecycle. The offline evaluation suite requires adopting teams to provide ground truth datasets. When these datasets are put into the product, they produce traces and spans that can be instrumented either through self-service or via BlackRock’s portal. These traces and spans undergo evaluation using various machine learning performance metrics including precision, recall, F1 score, Rouge score, and Jaccard score. The offline evaluation can be run on demand or on schedule, providing flexibility for different development workflows.
The online evaluation pipeline presents more challenges because ground truth is not available in real-time production scenarios. BlackRock’s current approach, which they acknowledge is not perfect, involves sampling outputs in real-time and routing them to human reviewers. These reviewers, working with labeled data, perform real-time computation of precision, recall, and other metrics. This hybrid human-in-the-loop approach balances the need for accuracy with the constraints of production environments.
The comprehensive nature of this evaluation infrastructure—covering both offline development cycles and online production monitoring—demonstrates a mature approach to LLMOps that goes beyond basic deployment practices.
The responsible AI product includes observability, telemetry, and surveillance capabilities across all guardrails and evaluation pipelines. This spans the end-to-end lifecycle of any AI product built at the firm, providing visibility into how AI systems behave from development through production. While specific technical details about the observability stack were not provided, the emphasis on comprehensive monitoring aligns with the overall philosophy that responsible AI must be treated as a first-class product concern rather than an afterthought.
While the primary focus was on responsible AI, the speaker provided context about the broader agentic AI platform being built at BlackRock. This multi-year project, still far from completion, spans all of BlackRock’s data domains including trading, risk, compliance, post-trade accounting, and more. Each domain has its own agents and multi-agents depending on the workflows being addressed. Critically, this agentic platform is being built around the responsible AI product, not the other way around. The responsible AI system sits at the center with the agentic capabilities built as connective tissue around it.
This approach contrasts sharply with typical development patterns where core functionality is built first and safety features are added later. By making the agentic platform plug into the mandatory responsible AI product, BlackRock ensures that governance and trust are foundational rather than supplemental concerns.
A significant aspect of BlackRock’s LLMOps approach is the deep integration with legal and compliance teams. The speaker described this relationship’s evolution from difficult courtship to strongest alliance. Initially, as a product manager and engineer, working with lawyers seemed daunting—lawyers would come up with ten additional edge cases for every two if-else statements in the code. However, this relationship has transformed to the point where legal and compliance teams are now shifted left in the development cycle.
Rather than joining at the end to provide a rubber stamp for completed AI products, legal and compliance now participate at the PRD (Product Requirements Document) level. This early involvement fundamentally changes the development process, ensuring that regulatory and compliance considerations shape product design from inception rather than constraining or blocking deployments after significant investment.
This shift is reflected in the key metric the team tracks: velocity to confidence, defined as the time between engineering complete and compliance sign-off. This metric improved dramatically from eight weeks to two weeks—a 75% reduction. This improvement occurred not because legal and compliance reduced their standards, but because they became allies with shared understanding of the requirements, participating in sprint-level activities rather than conducting lengthy post-hoc reviews.
BlackRock’s approach to measuring success for their responsible AI product is notably different from typical product metrics. The primary success criterion is that nothing happens—no regulatory incidents, no trust breaches, no hallucinations reaching users. This negative metric reflects the nature of risk management and governance in highly regulated industries where the goal is preventing bad outcomes rather than maximizing positive metrics.
Beyond the primary “nothing happens” criterion, the velocity to confidence metric provides a more actionable measure of the product’s effectiveness. By reducing the time from engineering completion to compliance approval from eight weeks to two weeks, the responsible AI product demonstrates its value in accelerating safe deployment rather than merely gatekeeping.
The philosophical framing that closes the presentation emphasizes that while there are eight technical guardrails, the speaker considers themselves and each team member to be an additional guardrail. This human element—the culture of responsibility and the understanding that upholding client trust requires personal accountability—represents a recognition that technical controls alone are insufficient. The combination of rigorous technical guardrails, comprehensive evaluation pipelines, deep legal integration, and a culture of personal responsibility creates a holistic approach to responsible AI deployment.
While BlackRock’s approach is comprehensive and appropriate for their regulatory environment, it’s worth considering the tradeoffs and broader applicability. The decision to make responsible AI the product itself, with other components as connective tissue, likely slows initial development compared to organizations that build first and add safety later. This is an intentional tradeoff where BlackRock prioritizes trust and regulatory compliance over development velocity.
The investment in custom guardrails like the investment advice detector represents significant effort that may not generalize to other industries. Organizations in less regulated environments might find such investment disproportionate to their risk profile. However, for financial services firms managing trillions in assets and facing potential SEC enforcement, this investment is clearly justified.
The online evaluation approach using human reviewers for sampling acknowledges a practical limitation—they don’t have a fully automated solution for production evaluation without ground truth. This hybrid approach balances accuracy with feasibility but does introduce human bottlenecks and potential scaling challenges as AI usage grows across the firm.
The claim to be potentially the only firm with an investment advice guardrail should be viewed with appropriate skepticism—other large financial institutions are likely working on similar problems, though possibly with different approaches or less public disclosure. The patent application suggests they believe their specific implementation is novel, but the broader problem of distinguishing advisory content from informational content is surely being addressed industry-wide.
The dramatic improvement in velocity to confidence from eight weeks to two weeks is impressive, but it’s worth noting this still represents two weeks of compliance review even with deep integration. Organizations in less regulated industries achieve much faster deployment cycles, highlighting that even optimized processes in highly regulated environments remain considerably more deliberate than in other sectors.
BlackRock’s case study represents a mature, production-grade approach to LLMOps in highly regulated environments. The architectural decision to treat responsible AI as the central product rather than a feature or layer reflects hard-won lessons about the risks of deferring safety considerations. The comprehensive guardrail framework aligned with NIST standards, combined with offline and online evaluation pipelines and deep legal integration, creates multiple overlapping layers of protection appropriate for an environment where hallucinations could have severe regulatory and financial consequences.
The case study demonstrates that responsible AI deployment in highly regulated industries requires not just technical controls but also organizational changes, including shifting legal and compliance teams left in the development process and cultivating a culture where individuals see themselves as guardians of trust. The multi-year timeline for the broader agentic AI platform, still incomplete, underscores the complexity of deploying advanced AI systems in environments where the cost of failure extends beyond product metrics to regulatory enforcement and institutional client trust.
London Stock Exchange Group (LSEG) Risk Intelligence modernized its WorldCheck platform—a global database used by financial institutions to screen for high-risk individuals, politically exposed persons (PEPs), and adverse media—by implementing generative AI to accelerate data curation. The platform processes thousands of news sources in 60+ languages to help 10,000+ customers combat financial crime including fraud, money laundering, and terrorism financing. By adopting a maturity-based approach that progressed from simple prompt-only implementations to agent orchestration with human-in-the-loop validation, LSEG reduced content curation time from hours to minutes while maintaining accuracy and regulatory compliance. The solution leverages AWS Bedrock for LLM operations, incorporating summarization, entity extraction, classification, RAG for cross-referencing articles, and multi-agent orchestration, all while keeping human analysts at critical decision points to ensure trust and regulatory adherence.
Nubank, one of Brazil's largest banks serving 120 million users, implemented large-scale LLM systems to create an AI private banker for their customers. They deployed two main applications: a customer service chatbot handling 8.5 million monthly contacts with 60% first-contact resolution through LLMs, and an agentic money transfer system that reduced transaction time from 70 seconds across nine screens to under 30 seconds with over 90% accuracy and less than 0.5% error rate. The implementation leveraged LangChain, LangGraph, and LangSmith for development and evaluation, with a comprehensive four-layer ecosystem including core engines, testing tools, and developer experience platforms. Their evaluation strategy combined offline and online testing with LLM-as-a-judge systems that achieved 79% F1 score compared to 80% human accuracy through iterative prompt engineering and fine-tuning.
Two organizations operating in highly regulated industries—Sicoob, a Brazilian cooperative financial institution, and Holland Casino, a government-mandated Dutch gaming operator—share their approaches to deploying generative AI workloads while maintaining strict compliance requirements. Sicoob built a scalable infrastructure using Amazon EKS with GPU instances, leveraging open-source tools like Karpenter, KEDA, vLLM, and Open WebUI to run multiple open-source LLMs (Llama, Mistral, DeepSeek, Granite) for code generation, robotic process automation, investment advisory, and document interaction use cases, achieving cost efficiency through spot instances and auto-scaling. Holland Casino took a different path, using Anthropic's Claude models via Amazon Bedrock and developing lightweight AI agents using the Strands framework, later deploying them through Bedrock Agent Core to provide management stakeholders with self-service access to cost, security, and operational insights. Both organizations emphasized the importance of security, governance, compliance frameworks (including ISO 42001 for AI), and responsible AI practices while demonstrating that regulatory requirements need not inhibit AI adoption when proper architectural patterns and AWS services are employed.
