## Overview Chevron Phillips Chemical (CP Chem) provides an insightful case study into how a large enterprise in the petrochemical industry is approaching generative AI adoption with a measured, governance-first strategy. Brent Riley, the Chief Data and Analytics Officer, describes the company's journey from forming their first data science team to consolidating data science, data engineering, and traditional business intelligence and analytics into a unified organization. This interview offers a window into how established industrial companies are navigating the generative AI landscape while balancing innovation with risk management. ## Organizational Response to Generative AI When the buzz around generative AI emerged, CP Chem took a structured approach by forming a cross-functional team to identify their approach to the technology. This team notably included stakeholders from legal and intellectual property, IT security, digital workplace, data science, data engineering, and analytics. This broad representation reflects an understanding that generative AI adoption is not purely a technical challenge but involves legal, security, and operational considerations that require enterprise-wide coordination. A key early priority for this team was to educate the leadership team on the technology and help them "cut through the hype." Riley emphasizes the importance of providing a realistic picture of LLM capabilities, acknowledging that while these tools can help accomplish things that were previously extremely difficult, they are not a "magic pill." This balanced perspective is refreshing in an industry often characterized by overblown claims and is indicative of a mature approach to technology adoption. ## Use Cases Under Exploration The use cases being explored at CP Chem are described as "very typical" for enterprise generative AI adoption, which provides useful benchmarking for other organizations at similar stages: **Virtual Agents on Specific Topics**: The company is developing virtual agents that can perform better than traditional chatbot technologies on specific subject matters. This suggests a focus on domain-specific knowledge retrieval and conversation rather than general-purpose chat interfaces. **Document Processing and RPA Integration**: A significant use case involves applying LLMs in the RPA (Robotic Process Automation) space for data processing tasks. The specific challenge addressed is handling unpredictable, unstructured information within documents such as PDFs, where traditional code-based extraction fails due to too much variation in source material. LLMs can help impose structure on this content and extract information that was previously inaccessible through conventional automation. This has applications in market intelligence processing and internal documentation analysis. **Operations Manual Virtual Assistant**: CP Chem possesses massive amounts of citations and operations manuals that represent a rich source of institutional knowledge. They are exploring how to build virtual assistants around these materials, deliberately starting with lower-risk applications. This incremental approach to deployment is a prudent strategy for managing risk while gaining experience with the technology. ## Model Strategy and Bias Considerations Rather than building models from scratch, CP Chem is taking a pragmatic approach by exploiting provided models such as Dolly from Databricks or GPT from OpenAI. This leverages existing capabilities while reducing the technical burden and time-to-value for their use cases. Riley provides thoughtful commentary on bias, noting that the term has multiple meanings that converge in the context of large language models. There is the technical data science definition of bias and the colloquial understanding, and both are relevant concerns with LLMs. Their approach to addressing bias focuses on several key strategies: - **User Training**: Helping users understand that bias is an inherent behavior of these models, not a bug to be fixed once and forgotten - **Prompt Engineering Education**: Teaching users how to prompt effectively and providing real-world examples for them to follow - **Setting Expectations**: Advising users from a research standpoint that LLMs should not be treated as a source of truth This emphasis on human-in-the-loop awareness rather than purely technical solutions reflects a realistic understanding of current LLM limitations. ## Testing Challenges One of the most significant operational challenges highlighted is testing. Riley candidly acknowledges that when building applications with open-ended user interfaces, traditional testing approaches are insufficient. The key questions they are grappling with include: - How do you test LLM-based applications that have open-ended interfaces? - How do you "harden" applications so they stay on topic as much as possible? - How do you prevent models from answering questions or performing actions outside their intended scope? These are fundamental LLMOps challenges that many organizations face when moving from proof-of-concept to production deployments. The interview does not detail specific solutions implemented, suggesting this may still be an area of active exploration for the organization. ## Data Governance and Platform Infrastructure CP Chem operates as a "very strong self-service shop," meaning they need solid, scalable platforms to serve data to their internal customers. They are a Databricks customer and have had significant success with this platform approach. The features provided by Unity Catalog are specifically mentioned as enhancing their ability to govern data at scale. From a policy and governance standpoint, they recently completed their generative AI policy, which focuses on several key areas: - **Appropriate Use Definition**: Establishing clear guidelines for what constitutes acceptable use of generative AI - **Risk Assessment**: Understanding and documenting the risks associated with different use cases - **Testing and Accountability**: Ensuring mechanisms exist to test for and account for identified risks - **Productivity Enhancement Focus**: Positioning generative AI primarily as a tool for improving productivity - **Traceability and Tractability**: Building in the ability to trace decisions and outputs back through the system This policy-first approach before widespread deployment reflects enterprise-grade thinking about AI governance and positions the organization to scale responsibly. ## Critical Assessment While this case study provides valuable insights into enterprise AI strategy, it is important to note several limitations in the information presented: The discussion remains largely at the strategic and conceptual level, with limited details on specific technical implementations, production deployments, or measurable results. The use cases described are still in exploration or early testing phases rather than mature production systems. This is not unusual for organizations in the early stages of generative AI adoption, but readers should understand that this represents a journey in progress rather than proven outcomes. Additionally, the interview format with the CDAO naturally emphasizes the strategic and positive aspects of the initiative. Specific challenges, failures, or lessons learned from attempted deployments are not discussed in detail beyond the general testing challenge. The reliance on third-party models (Dolly, GPT) while leveraging enterprise data raises questions about data privacy, model hosting, and integration that are not addressed in this transcript. Organizations considering similar approaches should carefully evaluate these considerations in their own contexts. ## Key Takeaways for LLMOps Practitioners This case study illustrates several best practices for enterprise generative AI adoption: The cross-functional team approach ensures that technical capabilities are aligned with legal, security, and business requirements from the outset. Starting with lower-risk use cases allows for organizational learning before tackling mission-critical applications. Investing in user training and awareness programs addresses the human factors that are critical for successful AI adoption. A policy-first approach provides guardrails and accountability mechanisms before scaling. Leveraging existing data platforms (Databricks/Unity Catalog) and pre-trained models accelerates time-to-value while maintaining governance standards. For organizations in similar industries with large repositories of technical documentation and operational data, the patterns described here offer a practical framework for approaching generative AI adoption in a responsible and systematic manner.