## Overview This case study from Credal, an enterprise AI platform company, provides a meta-analysis of LLM adoption patterns across multiple enterprise customers, with a particular focus on regulated industries. Rather than describing a single implementation, it synthesizes observations from Credal's customer base to outline common adoption patterns, key use cases, strategic decisions, and operational challenges that enterprises face when deploying LLMs in production. The document is authored by Credal's co-founder and should be read with the understanding that it is partly promotional material for their platform, though it does contain substantive insights into enterprise LLMOps challenges. Credal positions itself as a platform for secure generative AI deployment in the enterprise, offering features like access controls, audit logging, PII redaction, and multi-LLM support. The company began operations in summer 2022, predating the release of ChatGPT and GPT-4, which provides useful context for their perspective on the rapidly evolving LLM landscape. ## The Enterprise AI Adoption Curve The document presents a four-stage adoption model based on their observations of enterprise customers, using a background checking provider as an illustrative example: **Stage 1: Early Experimentation (AI Taskforce)** During this initial phase, organizations focus on learning and evaluation. There is typically an early spike of excitement with many users, which then levels off as the novelty wears off. Early adopters include CISOs, AI engineering leads, and some operations personnel. At this stage, organizations typically use vanilla chat interfaces like ChatGPT or Claude.ai without any connection to internal systems or data. **Stage 2: Chat with Docs Workflows** This stage begins after security audits have been passed and API integration has been established. It can be further divided into two sub-phases. First, organizations connect lower-security documentation to enable basic "Chat with Docs" workflows—for example, asking questions about HR benefits or searching GitHub issues. Usage broadens somewhat but remains relatively narrow. Subsequently, organizations begin integrating more sensitive documents (performance data, compliance questionnaires) as they develop firmer security policies, implement compliance checks, and establish rudimentary access controls. Audit logs, PII redaction, and acceptable use policies start being enforced at this point. **Stage 3: Enterprise Search** At this stage, users can ask any question that draws on company data and automatically receive responses from the most relevant sources. Access controls are rigorously enforced in near-real time across multiple data sources. This represents a significant operational maturity in terms of data governance and retrieval infrastructure. **Stage 4: Core Operations Workflows** This is where organizations move beyond chat-based interactions and integrate AI into core business processes. Examples include gathering feedback from sales calls, AML/KYC workflows, customer support automation, and receipt matching. Agent workflows, copilots, and LLM orchestration become prominent. AI capabilities expand beyond text completion to include code execution, image generation, and sophisticated data retrieval. ## Production Deployment Challenges The document identifies several key challenges that organizations face when moving LLM prototypes into production environments: **Privacy, Security and Compliance** When integrating company data with LLMs, organizations must address data governance concerns. The text emphasizes that data security is "the single biggest barrier to enterprise adoption of AI." Many public-facing end-user tools do not provide sufficient visibility for IT teams into what end users are doing. Organizations fear that employees are copy-pasting sensitive company data, including customer PII, into external LLM tools. This concern has led many organizations to either ban or block all usage, creating a tension between security requirements and productivity benefits. **Reliability and Quality Over the Long Tail** Maintaining high-quality performance across the full distribution of production queries presents ongoing challenges. The document notes that evaluation of LLM-based systems is "notoriously difficult," meaning that minor-seeming changes to a critical prompt can cause major changes in overall system behavior. This makes companies understandably wary of deploying LLMs into critical workflows without extensive testing, governance, and version control. **RAG Infrastructure Complexity** The document explicitly calls out that "search use cases are hard to productionize, debugging is difficult, infrastructure, governance, access management and such are super complicated components." This acknowledgment of RAG (Retrieval-Augmented Generation) challenges is particularly relevant for LLMOps practitioners who need to manage chunking strategies, embedding pipelines, retrieval mechanisms, and reranking systems. ## Strategic Decisions for Enterprise LLMOps The case study presents three key strategic decisions that enterprises must make: **Build vs. Buy** For core business processes, the document recommends building in-house and owning the logic. A mature fintech firm might build its own AML/KYC solutions to leverage proprietary data. However, for non-core use cases, procurement is recommended. The document notes that for products like call transcription and coding assistants, the market has mature offerings that will likely function as well as in-house solutions at a fraction of the cost. Similarly, enterprise search products are recommended for procurement rather than in-house development due to the high maintenance burden. A key consideration here is deciding what requires direct control versus what can be delegated. Questions include: Do you need an opinionated chunking strategy for your use case? What about reranking and retrieval strategy? The recommendation is that for core business functionality, organizations should own all decisions; for basic use cases, they should buy existing solutions. **Platform vs. Point Solutions** Credal advocates strongly for platform approaches over point solutions (unsurprisingly, given their business model). The argument is that models update rapidly—what was leading-edge yesterday may not be today—so organizations need something adaptable and customizable. The exception is certain concrete point solutions like GitHub Copilot for coding, where the market has established clear winners. For most other use cases, the recommendation is to buy a platform that makes it easy to configure and build AI solutions while handling operational concerns. An important consideration is that non-technical users should be able to use the platform for experimentation, not just developers. This democratization of AI tooling is seen as essential for discovering valuable use cases. **Single LLM Provider vs. Multi-LLM Strategy** The document strongly advocates for a multi-LLM strategy. An interesting observation is that smaller companies tend to be more willing to lock themselves into a single provider, while larger and more sophisticated organizations resist this approach. The document provides useful guidance on when to use different model tiers. For "90th-100th percentile" tasks requiring higher-level reasoning and coding—such as reading a GitHub issue and producing a pull request—frontier models like GPT-4 and Claude 3 remain superior. For "0th-90th percentile" tasks, open-source models like Mixtral-8x7B or LLaMA 2 can be fine-tuned to match or exceed GPT-3.5 performance and are more cost-effective for simpler cognitive tasks like classification and tagging. The document cites Ramp's receipt matching workflow as an example of using open-source models effectively. For specific tasks like SQL generation, open-source text-to-SQL models (such as Defog) are noted to outperform GPT-4 on benchmarks. ## Enterprise Strategy Observations The document identifies two main AI strategies that enterprises adopt: **Strategy 1: Sandbox Approach** Ban all external tooling and provide a specific sandbox for employees. Companies following this approach typically build internal wrappers around Azure OpenAI—usually a Slack/Teams integration that logs all queries for IT visibility. About 260+ companies chose ChatGPT Enterprise, but many also maintain their own Azure wrappers since ChatGPT Enterprise costs significantly ($40-$60 per user per month). The limitation is that just providing a chat UI without data integration does not significantly accelerate the business. **Strategy 2: Controlled Experimentation** Provide a set of tools experimentally that are integrated with company data, along with guidelines about appropriate use, and allow employees to discover valuable use cases. This approach yields more experimentation and innovation but requires solving numerous issues around security, access control, and governance. ## Adoption Metrics and Use Cases The document provides some adoption metrics: in tech companies, roughly 50% of the organization uses AI tooling within 8 weeks of adoption, reaching 75% after a year when implementations are successful. The authors acknowledge limited data beyond one year given the technology's novelty. Key production use cases observed include: **Engineering**: Coding assistants (GitHub Copilot, Sourcegraph Cody, Codeium) for productivity, plus LLMs powering core product functionality like receipt matching and automated employee background checks. **Operations and Finance**: AML/KYC automation, transaction monitoring, receipt matching, and accounting workflows. **Legal and HR**: Contract negotiations, privacy and compliance query handling, and HR benefits management. **Marketing and Sales**: Transcription, notes synthesis, and copywriting workflows. ## Additional LLMOps Challenges Beyond the core technical challenges, the document identifies several organizational and operational barriers: **Use Case Discovery and Value Quantification**: Understanding which use cases deliver actual business value is challenging, especially at scale where AI tool costs ($20-$60 per seat) must be justified across organizations. There is concern that "hype" outpaces actual utility. **Duplicate Use Cases and Data Fragmentation**: Multiple teams may build similar workflows in different tools, creating fragmented data landscapes and additional costs. **Legal and Regulatory Barriers**: Concerns about the NY AI Act, EU AI Act, and compliance requirements create uncertainty. There is a general lack of understanding about what these regulations actually require. **Human Resources Challenges**: Employee concerns about AI-driven job displacement and the need for learning and development programs to equip employees with necessary skills. ## Critical Assessment It is important to note that this document serves partly as promotional material for Credal's platform. The strong advocacy for platform approaches over point solutions, the emphasis on multi-LLM strategies, and the focus on security and access control challenges all align with Credal's product positioning. While the observations about enterprise adoption patterns and challenges appear substantive and align with broader industry discourse, readers should be aware of the commercial context. The document provides useful frameworks for thinking about enterprise LLMOps but should be balanced against perspectives from other vendors and practitioners in the space.