## Overview This case study captures a panel discussion at Google Cloud Next featuring data governance leaders from multiple industries discussing how generative AI is accelerating the need for robust data governance while simultaneously providing new tools to automate governance processes. The panel includes representatives from Collibra (a data governance platform partner), Mercado Libre (Latin America's largest e-commerce platform), ATB Financial (a Canadian regional bank), and Loblaw (Canada's largest retailer). The discussion provides valuable insights into how organizations across different sectors are approaching the intersection of LLMOps and data governance. ## The Dual Impact of GenAI on Data Governance The panel highlights a critical dynamic in the current enterprise landscape: generative AI creates enormous demand for well-governed data while simultaneously offering new capabilities to automate governance tasks. As the moderator from Google Cloud's Dataplex team notes, "data is really the foundation for AI" and organizations must ensure their data is properly governed before using it to train or ground AI models. At the same time, GenAI enables "leap frog opportunities" to build more intelligent governance capabilities. ## Company-Specific Approaches and Use Cases ### Collibra's Perspective on AI-Enhanced Governance Alexander from Collibra, who has been in the data governance space for over 15 years, emphasizes that generative AI is transforming what was traditionally a highly manual process. He identifies two key pillars where GenAI is changing governance: The first pillar is automation of previously manual tasks. Description generation is a prime example—where data stewards previously had to manually document columns and tables, GenAI (specifically Gemini) can now propose descriptions automatically. Users simply need to review and save rather than create from scratch. This represents a fundamental shift in the effort required for data cataloging. The second pillar is ensuring AI-ready data for production AI systems. Alexander emphasizes the maturity levels of AI implementation, from simple prompt engineering to RAG (Retrieval-Augmented Generation) systems that add organizational data on top of foundation models. When implementing RAG, data quality becomes critical because "whatever small amount of bad data that is going to get in the model will have repercussions." He cites the infamous Chevrolet chatbot example where poor grounding led to the bot recommending customers buy Teslas instead—a stark illustration of why data governance matters for production AI systems. Collibra has also launched AI governance capabilities, with their first integration being Vertex AI, allowing all Vertex AI assets to automatically flow into their governance platform. Their GenAI features, including description generation, are powered by Gemini. ### Mercado Libre's Data Mesh and GenAI Integration Barbara from Mercado Libre describes an organization processing over 300 transactions per second with 54 million quarterly buyers—true big data at scale. Their governance journey is tightly integrated with their data mesh adoption three years prior. The data mesh paradigm enabled thousands of users to publish data products in a self-service manner, moving away from a centralized bottleneck model. Governance rules, procedures, and best practices were embedded directly into the technology platform, making it easier for non-specialist teams to follow governance protocols. For GenAI initiatives in production, Mercado Libre is leveraging their years of data cataloging investment. Barbara notes that stewards who spent years documenting assets are now seeing the payoff as this structured knowledge can be used to train and ground LLMs. Specifically, they are building an internal agent to answer common questions that users typically bring to the data and analytics team—questions like "How do I request a license for this product?" or "What does this metric mean?" or "Where can I find a dashboard with this metric?" This knowledge was previously locked in silos but is now being democratized through RAG-based approaches grounded on their BigQuery data and internal documentation. With 1,300 BigQuery projects and 60,000 employees, knowledge transfer is critical. Only 300 projects belong to the core team—the other 1,000 are teams across the company creating and publishing data products. They've cultivated data champions across the organization through partnership with Google's learning team. ### ATB Financial's Cautious, Methodical Approach Jeff from ATB Financial represents a regulated financial institution taking a more measured approach to GenAI adoption. Their philosophy is "slow as smooth, smooth as fast." They've established a cross-functional Generative AI Steering Committee that includes HR, business units, and various partners to consider how use cases impact the workforce of the future. Their approach involves tactically evaluating different tiers of generative AI use cases—some simple enough for Google Workspace, others requiring more sophisticated tools like Vertex AI. They're carefully aligning their product roadmap with Google's product evolution while focusing on change management and preparing leaders and teams for transformation. ATB recently migrated from an "ungoverned Legacy on-prem data" environment to Google BigQuery, establishing strong data governance foundations in the process. This migration involved empowering data owners and data stewards and represented a significant culture change. The rigorous governance foundation is now enabling strong data quality for consumers and providing a framework that they've extended to responsible AI and AI ethics practices, preparing for alignment with the ISO 42001 standard. They've recently introduced DLP (Data Loss Prevention) for BigQuery, Google Workspace, Gmail, and Chat, and the strong governance foundation made this a smooth transition. The security-first mindset is essential for a financial institution, and Google's end-to-end security capabilities help "reduce the risk and really keep data safe, secure, and reliable." ### Loblaw's Three Pillars of Governance and GenAI Exploration Uswell from Loblaw describes governance through three pillars: discoverability, trustability (including quality and observability), and security/compliance. They operate a Center of Excellence (CoE) model with hub-and-spoke implementation. Loblaw has their own internal GenAI product called "Garfield" with multiple models hosted in their own environment and a frontend interface. They're also exploring Vertex AI's search capability to help team members answer questions from internal documents without needing to reach a person directly. A significant governance concern for Loblaw is governing RAG implementations at scale. They're considering questions like: If a store should not see other stores' sales numbers, how do you enforce that in a RAG context? Is Active Directory sufficient, or do deeper restrictions need to be implemented? They're also concerned about prompt-level privacy—if someone prompts with PII, what happens to that information? Uswell emphasizes that while AI algorithms are "extensions of mathematics—if they work, they work"—the real challenge is data. Poor quality data with errors and biases will creep into model biases. For regulated industries, this is critical: if AI is used for loan approval decisions, organizations need explainability to understand why person A was approved and person B was not. Their journey with Dataplex spans over two to three years as early development partners. They value the 360-degree visibility across all GCP projects and appreciate the end-to-end support for tools like GCS buckets, BigQuery, Composer, and Dataproc. The recently announced column-level lineage capability was specifically called out as important—a feature developed in response to customer feedback. ## Key Technical Capabilities Enabling LLMOps Several Google Cloud capabilities were highlighted as critical for production LLM deployments: **Dataplex** provides unified metadata across distributed data with global search, centralized security and governance, data profiling, data quality monitoring, and lineage tracking. The platform now offers column-level lineage (in preview) and semantic search for natural language interaction with data catalogs. **BigQuery** serves as the analytics foundation for multiple organizations, with Mercado Libre running 1,300 projects and Loblaw maintaining presence across numerous GCP projects. **Vertex AI** integration enables AI asset governance through partnerships with Collibra, with assets automatically flowing into governance platforms. **Semantic Search** capabilities are democratizing data discovery by allowing natural language queries rather than requiring users to learn search syntax. Instead of searching for "customer table" and getting 20 results to manually evaluate, users can describe what they need semantically and receive targeted results. ## Governance Considerations for Production LLMs The panel identified several governance challenges specific to LLMOps: - **Data Quality for Grounding**: Ensuring high-quality data for RAG implementations is critical because small amounts of bad data will have downstream effects on model outputs - **Model Observability**: Tracking model performance over time as models deteriorate and may not produce the same outputs they did at launch - **Version Control**: Implementing version control for ML models - **Ethical and Legal Compliance**: Ensuring AI usage is ethical, legal, and compliant - **Access Control in RAG**: Determining how to enforce data access restrictions when data is being retrieved and injected into prompts - **Prompt Privacy**: Managing PII exposure through prompts - **Explainability**: For regulated industries, understanding and explaining why models make specific decisions ## The Partnership Model A recurring theme is the value of close partnerships between customers and vendors. Multiple panelists praised the bidirectional feedback loop with Google Cloud, noting that customer input directly shapes product roadmaps. The Collibra-Google partnership was highlighted as enabling multicloud approaches with tight integrations between governance platforms and AI capabilities. The overall message is that data governance has evolved from being a roadblock to becoming an enabler for AI initiatives, but only when organizations invest in solid foundations first.