Burns & McDonnell, a global architectural engineering and construction company, deployed a multi-agent system called "Experience IQ" to solve the challenge of matching employees with complex skill requirements across diverse projects and locations. Built using Google Cloud's Agent Development Kit (ADK) and deployed through Gemini Enterprise App, the system leverages historical data stored in Spanner, generates SQL queries from natural language, and uses multiple specialized agents with callbacks and routing logic to find qualified personnel. The solution successfully replaced tribal knowledge and disconnected systems with an automated, production-ready agent platform that pairs the right people with the right projects, improving project outcomes and client satisfaction.
Burns & McDonnell is a 100% employee-owned architectural engineering and construction company operating across multiple industries and geographies. The company faces a critical operational challenge: they execute projects requiring people with specific skill combinations, but their workforce is distributed across many locations with knowledge residing in disconnected systems. Prior to their AI solution, the company relied on tribal knowledge, email chains, and fragmented systems of record to match people with project requirements—a manual, inefficient approach that couldn’t scale with their diverse project portfolio.
The company built “Experience IQ,” a multi-agent system deployed to production using Google Cloud’s Agent Development Kit and Gemini Enterprise App. This case study is particularly valuable because it demonstrates real production deployment of agentic systems in an enterprise environment, complete with governance, evaluation, and scalability considerations.
The presentation situates Burns & McDonnell’s implementation within Google Cloud’s comprehensive agentic platform strategy, which addresses three major challenges observed across 70+ customers in their global customer partnership program:
Google’s research found that six months prior, 90% of customers were blocked or at risk on governance challenges specifically. The platform addresses this through a “build, scale, governance, and optimize” framework.
The presentation highlights three critical shifts observed in production agent deployments:
From monolithic to mesh topology: Organizations are moving from single large agent applications to flexible mesh architectures where subagents can be decentralized across departments, called as needed, and reused across the company. This enables better modularity and team autonomy.
From tools to skills: While tools remain important, skills have emerged as a critical abstraction that ensures the right tool is called at the right time. Skills can be written in natural language by domain experts, bringing tribal knowledge into agent context while minimizing latency, reducing context bloat, and lowering costs.
From probabilistic to deterministic workflows: Production systems are shifting away from relying solely on prompt engineering and instructions to control agent behavior, moving toward embedding business logic and compliance rules at the framework layer. This provides the safety, legal compliance, and human-in-the-loop controls that production systems require.
The ADK represents Google’s synthesis of best features from existing frameworks like LangGraph and Crew, designed specifically for building multi-agent systems. Key differentiators include session management, multi-modality support, and continuous feature releases.
The development workflow includes:
The framework supports multiple agent orchestration patterns including sequential agents (guaranteed execution order), loop agents, parallel agents, and router agents with custom prompting. This flexibility allows developers to choose deterministic flows where needed while maintaining the flexibility of LLM-based routing elsewhere.
Once developed, agents can be deployed to various runtimes (laptops, servers, Cloud Run, Kubernetes), but standard runtimes lack agent-specific capabilities. Google’s Agent Platform Runtime provides a specialized deployment target that automatically includes:
Deployment requires only two lines of code: wrapping the root agent in an ADK app and creating an Agent Engine with configuration. The system handles containerization, deployment, and infrastructure provisioning automatically, taking approximately five minutes.
The deployed agents provide extensive production visibility through the Agent Platform interface:
This observability is critical for the “hill climbing on agent quality” challenge—providing the quantitative foundation for systematic improvement.
The platform includes comprehensive governance tools released as part of the agent platform:
Agent Identity: Each agent receives a unique identity that can be tracked throughout its end-to-end lifecycle, enabling auditability and accountability.
Agent Registry: A centralized repository where all deployed agents are cataloged, enabling discovery and reuse across the organization. This directly addresses the agent sprawl problem by providing a “single pane of glass” for understanding what agents exist.
Agent Gateway: A centralized policy enforcement point that controls:
The gateway acts as the “centralized choke point” for deploying and enforcing policy—not glamorous work but essential for moving beyond prototypes to production.
Security scanning: The platform automatically scans deployed agents for vulnerabilities in dependencies, surfacing issues without requiring additional code or configuration from developers.
Google emphasizes protocol adoption for standardizing agent interactions:
This protocol-first approach is critical for enterprises with heterogeneous environments and long-term interoperability requirements.
The presentation provides a comprehensive reference architecture for enterprise agent platforms with distinct environments:
Platform team responsibilities: Creating projects, networking, IAM roles and policies, providing engineers with development environments
Data environment: A data lake implemented in a rug (likely referring to a data architecture pattern) linked to development environments
Development environment: Extended for backends, frontends, and agents-as-a-service, with a governance layer providing an “AI control tower”
Staging environment: For testing agents before production promotion, following the classic dev-staging-prod pipeline
Production environment: With augmentation for logging and long-term memory, recognizing that agents are stateful unlike traditional stateless applications
Exposure layer: Websites like Gemini Enterprise App where agents are made available to end users
This architecture separates concerns appropriately while maintaining the statefulness and observability requirements unique to agent systems.
Burns & McDonnell’s implementation begins with data preparation. The company ingested historical data from multiple systems into Google Spanner as their database of choice. Critically, they performed enrichment using Gemini before ingestion and generated embeddings to support semantic search and retrieval.
This data foundation is essential—as emphasized throughout the presentation, agent quality depends fundamentally on data quality. The team explicitly chose Spanner, likely for its strong consistency guarantees and global distribution capabilities.
The Experience IQ system implements a sophisticated multi-agent architecture with specialized roles:
Experience IQ Agent (coordinator): Greets users and orchestrates the overall interaction
Callback-based retrieval: The system uses ADK’s callback functionality to retrieve:
Router agent: Makes routing decisions to either query the entire database or a subset based on query characteristics and retrieved context
Schema narrowing agent: When routing to the entire database, this subagent narrows down which schemas are relevant for SQL generation, reducing complexity and improving accuracy
SQL code generation agent: Generates SQL queries from natural language and executes them directly against Spanner
Retry logic: If SQL execution fails, the system refines the query up to three times before returning an error
An important lesson emerged after initial deployment: the agent was producing technically correct SQL that wasn’t business-friendly. This highlights a common production challenge where technical correctness doesn’t equal user satisfaction.
The team’s solution demonstrates mature LLMOps practices:
This represents a sophisticated application of preference learning and evaluation-driven improvement—exactly the “hill climbing on agent quality” capability that was identified as a major challenge.
Burns & McDonnell explicitly chose natural language to SQL as their approach but architected the system so this component could be swapped out for different approaches without changing the underlying agent deployment infrastructure. This demonstrates good engineering—separating the domain-specific logic (SQL generation) from the platform concerns (deployment, governance, monitoring).
Rather than creating hypothetical test scenarios, the team put the system in the hands of actual users and instructed them to ask questions they would normally only be able to answer through manual research. This production-first evaluation approach provides the most realistic assessment of system value.
The demonstration showed a complex query successfully returning a specific employee (Andrew Stearns himself) who matched multiple skill criteria. The system successfully replaced tribal knowledge, email chains, and disconnected systems with a unified interface.
The business outcomes are clear: pairing the right people with the right skills on the right projects leads to better project outcomes, happier employees, and satisfied customers—ultimately improving the company’s financial performance.
Burns & McDonnell views this as the starting point of their agent journey. Their future state envision a single interface that can handle any client request—whether a request for proposal or a simple question—by finding the right people and building complete teams automatically. This represents a progression from information retrieval to action and orchestration.
The presentation included several live demonstrations that illustrate the developer experience:
The demo showed building a math tutoring agent for kids with multiple specialized subagents:
Each tool is defined as a simple Python function with a detailed docstring describing its purpose, inputs, outputs, and examples. This documentation becomes the agent’s understanding of when and how to use the tool.
The sequential agent pattern guarantees execution order: grammar check first, then math operations, then summary. This deterministic flow ensures appropriate behavior for the use case.
A demonstration showed a voice-enabled agent that could:
This demonstrates ADK’s native multi-modality support, going beyond text-only interactions.
The most striking demonstration showed using natural language to generate a complete booking agent in under a minute:
“Could you create for me a booking agent to help the user book holidays. Use the development style in this code repo and add the new agent under agents folder.”
The system:
This “vibe coding” approach dramatically accelerates agent development by allowing developers to describe what they want rather than implement every detail.
While the presentation demonstrates impressive capabilities, several considerations merit attention:
Vendor-specific lock-in: The solution is deeply integrated with Google Cloud’s proprietary stack (ADK, Agent Platform Runtime, Gemini). While protocols like MCP provide some standardization, migrating to another platform would require significant re-engineering. Organizations should weigh this against the substantial productivity benefits.
SQL generation risks: Natural language to SQL is a well-known challenge area. While Burns & McDonnell’s approach of using historical examples and subject matter expert annotations is sound, SQL generation can produce incorrect queries, security vulnerabilities (SQL injection), or queries that don’t match user intent despite being syntactically valid. The three-retry mechanism helps but doesn’t eliminate these risks.
Evaluation challenges remain: Despite the extensive monitoring and observability, the presentation acknowledges that evaluation is still a “dark art.” The simulator for generating golden datasets is helpful, but determining what constitutes “good” agent behavior for complex multi-turn interactions across diverse domains remains challenging.
Governance overhead: The comprehensive governance infrastructure (registry, gateway, identity) addresses real needs but introduces operational complexity. Organizations need sufficient scale and compliance requirements to justify this overhead—it may be overkill for smaller deployments.
Production maturity: While the demonstrations are impressive, the maturity of the Agent Platform Runtime in production at scale isn’t fully established. Early adopters may encounter issues that haven’t been discovered yet.
Cost implications: The presentation doesn’t address cost—running multiple agents with multiple LLM calls, embeddings generation, Spanner queries, and comprehensive observability likely incurs substantial cloud costs. Organizations should model expected usage carefully.
That said, Burns & McDonnell’s implementation demonstrates genuine production deployment with real business value. The comprehensive platform approach addresses authentic pain points observed across many enterprises, and the governance-first mindset is appropriate for regulated industries and large organizations.
This case study offers several valuable lessons:
Data foundation is paramount: Burns & McDonnell’s success started with comprehensive data ingestion, enrichment, and embedding generation before building agents.
Multi-agent architectures scale better than monoliths: The specialized subagent approach with clear routing logic provides better modularity and maintainability than single large agents.
Production requires governance from day one: The registry, gateway, and identity infrastructure aren’t afterthoughts—they’re essential for moving beyond prototypes.
Evaluation must be business-driven: Technical correctness isn’t sufficient; subject matter expert validation and preference learning are necessary to align agent behavior with business needs.
Deterministic workflows where possible: Using sequential agents and embedded business logic reduces reliance on probabilistic prompt engineering for critical paths.
Observability must be comprehensive: The detailed traces, metrics, and logs enable the systematic improvement necessary for production systems.
Developer experience matters: The rapid development capabilities demonstrated (Agent CLI, ADK Web) reduce time-to-value and lower barriers to entry for agent development.
The Burns & McDonnell case study represents a mature approach to production agent deployment in a complex enterprise environment, demonstrating that with appropriate platform infrastructure, governance, and evaluation practices, multi-agent systems can deliver tangible business value in production settings.
Snorkel developed a specialized benchmark dataset for evaluating AI agents in insurance underwriting, leveraging their expert network of Chartered Property and Casualty Underwriters (CPCUs). The benchmark simulates an AI copilot that assists junior underwriters by reasoning over proprietary knowledge, using multiple tools including databases and underwriting guidelines, and engaging in multi-turn conversations. The evaluation revealed significant performance variations across frontier models (single digits to ~80% accuracy), with notable error modes including tool use failures (36% of conversations) and hallucinations from pretrained domain knowledge, particularly from OpenAI models which hallucinated non-existent insurance products 15-45% of the time.
Notion, a knowledge work platform serving enterprise customers, spent multiple years (2022-2026) iterating through four to five complete rebuilds of their agent infrastructure before shipping Custom Agents to production. The core problem was enabling users to automate complex workflows across their workspaces while maintaining enterprise-grade reliability, security, and cost efficiency. Their solution involved building a sophisticated agent harness with progressive tool disclosure, SQL-like database abstractions, markdown-based interfaces optimized for LLM consumption, and a comprehensive evaluation framework. The result was a production system handling over 100 tools, serving majority-agent traffic for search, and enabling workflows like automated bug triaging, email processing, and meeting notes capture that fundamentally changed how their company and customers operate.
Union's Chief ML Engineer shares lessons learned from productionizing agentic systems at scale, addressing the critical infrastructure challenges that arise when deploying LLM agents in production environments. The presentation introduces six design principles for building crash-proof, durable agents using the Flyte 2.0 orchestration platform, focusing on how agents can recover from multi-layer failures (infrastructure, network, logical, semantic) through proper context engineering and durability mechanisms. A key case study with Dragonfly demonstrates these principles in action, where a tiered agent architecture processes 250,000+ software products with 200+ steps and 100+ LLM calls each, achieving 2,000+ concurrent runs, 50% reduction in failure recovery time, 30% increased development velocity, and 12 hours per week saved on infrastructure maintenance.
