## Overview and Context This case study emerges from a panel discussion at an AWS conference featuring multiple perspectives on deploying LLMs in production across private equity portfolio companies. The speakers include Sanjay Subramanyan (PwC partner leading cloud, data and AI), Chai from Warburg Pincus (operating partner focused on AI transformation), Robbie from Abrigo (CTO/CPO of a Carlyle portfolio company serving 2,500+ community banks), Chad Burnick (AWS principal solutions architect for PE), and Nate Barnes (PwC chief data scientist for deals practice). The discussion reflects on the three-year journey since ChatGPT's launch in November 2022, providing insights into what actually worked versus what was hyped in bringing LLMs to production. Abrigo's specific context is particularly illuminating: they operate in the regulated banking space, providing intelligent workflow software for fraud detection, anti-money laundering, and loan origination to community banks and credit unions. As an innovation partner to institutions that lack the budgets of major banks like JPMorgan, Abrigo must stay ahead technologically while operating under strict regulatory constraints. This makes their production AI journey especially instructive for understanding the practical challenges of deploying LLMs in regulated, high-stakes environments. ## Initial Expectations and Reality (2022-2023) When ChatGPT launched in November 2022, the private equity sector's initial response was characterized by excitement mixed with uncertainty. At Warburg Pincus, while foundation models and LLMs were already being monitored from an investment angle, the surprise came from the pace of adoption rather than the technology itself. This triggered numerous questions across multiple dimensions: Would it impact existing portfolios? Did due diligence processes need reinvention? How would investment theses need to change? The firm's thoughtful response involved establishing small teams to explore internal usage, beginning to incorporate AI considerations into investment thesis writing, and initiating conversations with software portfolio company CEOs about potential impacts. At Abrigo, the initial reaction combined awe with recognition of potential disruption. The company's advisory practice, which helps banks understand regulatory updates, suddenly faced a scenario where customers could simply upload documents to ChatGPT and ask questions directly. Within the first few months of 2023, despite being a technology company in a regulated space requiring careful vetting of new technologies, Abrigo began internal deployment. Their product teams quickly recognized opportunities in workflow spaces where narrative content was required—fraud alerts, credit memos, loan documentation—where AI could potentially reduce a 30-minute writing task to 10 minutes by providing intelligent starting points. ## Common Early Patterns and Misconceptions Chad from AWS observed consistent patterns across private equity portfolio companies in early 2023. Almost everyone attempted customer-facing chatbots first, followed quickly by summarization use cases—particularly summarizing data room contents and financial statements for investment decisions. Knowledge base extensions were also popular. However, a critical misconception emerged: companies believed that ChatGPT and custom GPTs could simply understand their messy, less-than-structured data and "figure it out." This assumption led to numerous stalled initiatives as the reality of data quality requirements became apparent. The more successful approaches, according to Chad, involved stepping back from technology-first thinking and identifying genuine high-friction business problems. One notable example that solved a real business need involved an e-commerce company with a large image catalog needing enrichment—multiple views, 3D representations, contextual placements. Rather than manual photography and processing, they used AI to generate synthetic images, completing in a fraction of the time what would have been a highly manual, resource-intensive process. While this was a one-off problem rather than an ongoing production system, it demonstrated the value of applying AI to specific friction points rather than pursuing chatbots because they were trendy. ## Key Success Factors for Production Deployment ### Business-First, Not Technology-First Nate Barnes emphasized that the fundamental differentiator between proof-of-concepts that stalled and those that reached production was starting with business value propositions rather than AI for its own sake. Too many organizations assigned AI problems to IT departments without thinking about business transformation first. The critical question framework should focus on where complexity prevents business growth into new customer bases, market expansion, or cost reduction—these are value questions, not AI questions. Those who prioritized understanding where high friction existed in valuable business processes, then experimented rapidly in those specific areas, achieved success. ### Leadership Alignment Over Technical Infrastructure Chai from Warburg Pincus repeatedly stressed that across their portfolio, the single most critical success factor was not data quality, technical capabilities, or talent—it was broad leadership alignment. Companies with poor data but aligned leadership made things happen through perseverance and overcoming challenges. Conversely, even with the best ideas, best products, and good technical foundations, initiatives failed without leadership buy-in. This led Warburg Pincus to change their prioritization approach: they now start with leadership alignment and won't engage on AI initiatives if leadership isn't aligned, regardless of the technical opportunity. This requires leadership training on the business side about what AI programs entail—understanding that success requires time, perseverance, and learning from failures. ### Rapid Experimentation Cycles The panel emphasized dramatically shortened experimentation cycles compared to traditional software development. Nate described moving from multi-week POCs to afternoon or 24-hour experiments. Robbie shared an example where a team was given two weeks to rebuild a legacy product that had been consistently estimated at 12-18 months for redevelopment. While they only achieved 50% completion in those two weeks, the product was slated for release in January after work begun in August—a dramatic acceleration. The key principle is that AI prototyping should enable very fast, very inexpensive failures, not multi-month exercises. ### Data Readiness and Pragmatic Approaches Chad highlighted that companies with mature data lake/lakehouse architectures—built not specifically for GenAI but to enable data-driven decision making across the organization—had significantly better outcomes when implementing LLM solutions. These organizations already had clean, accessible data, understood scaling challenges, and possessed teams experienced with data-centric thinking. For companies without this foundation, the recommended approach was pragmatic: select two or three high-value use cases, assess if the required data is close to production-ready for those specific cases, and implement piecemeal data governance focused on those use cases rather than attempting comprehensive data transformation programs that consume resources without demonstrating value. ## Production Use Cases and Implementation Details ### Workflow Automation in Banking Abrigo's production implementations focused on intelligent workflows within their loan origination, fraud detection, and anti-money laundering products. With usage analytics across 1,000+ banks using their loan origination system, they could identify precisely where 5-15 people touching each loan application spent time—both within the system and outside it. The goal was reducing friction at these specific points. For credit memo generation, which typically required 15-20 minutes, providing an AI-generated starting point saved approximately 10 minutes per memo. The explicit mission wasn't eliminating roles like credit analysts or underwriters, but rather reducing undifferentiated grunt work so community bank employees could spend more time on relationship building—their core differentiator—and better serve local businesses. The implementation philosophy balanced automation with workflow preservation. Rather than completely reimagining 30-year-old workflows and risking disruption, they identified where AI could provide power without forcing users to reimagine entire processes. An example involved document verification in loan applications: if a borrower submitted a 2023 W-2 when a 2024 form was needed, an AI assistant could immediately respond rather than waiting for a human data analyst to review it the next day, improving customer experience in an inherently indeterministic interaction. ### Engineering and Development Acceleration Robbie noted that for the first time in his 25-year career, engineering was no longer the bottleneck. Coding assistance tools dramatically increased code production speed. However, this introduced new challenges: senior engineers sometimes resisted because the generated code didn't match their aesthetic preferences or standards. This required organizational conversation about whether code needed to be optimized for human consumption or whether documentation-supported code readable by agentic systems was sufficient. The real transformation came from recognizing that with near-zero prototyping costs, the entire product development process needed reimagining. Previously, getting from idea to prototype required extensive documentation and wireframes over 6-8 weeks. Now, the approach shifted to "just show it"—build a working prototype in a weekend and demonstrate rather than document. This shifted bottlenecks leftward to UX and product management. How quickly could teams vet ideas when engineering could implement anything rapidly? The organizational challenge became maintaining pace across the entire product development lifecycle, not just engineering. Robbie emphasized the cultural change required: leaders needed hands-on understanding of tools like coding assistants to appreciate their power and push adoption downward through the organization while finding grassroots champions. ### Investment Diligence Transformation Nate described how AI diligence has evolved dramatically from 2022 to 2025. Initially, diligence questions focused on maturity and readiness—a back-office checkbox exercise covering data quality, technical stack, and people capabilities. By 2025, AI became an existential question in investment committees. The primary concern shifted to disruption risk: can two developers in a garage replicate this business? While often the answer is "probably," existing companies retain advantages through product moats, data moats, and customer relationships that can't be easily overcome. This mirrors how 15-20 years ago, every consumer packaged goods investment required answering "what if Amazon disrupts this?"—now every IC asks "what if AI disrupts this?" The diligence question evolved from a narrow IT enterprise maturity check to encompassing commercial diligence (market positioning relative to AI capabilities), operational questions (cost takeout potential in call centers, engineering teams), and product questions (competitive feature positioning). Every investment committee now requires a one-page assessment of both AI risks and opportunities. Companies must defend that they won't be disrupted within the 5-10 year exit horizon. AI is no longer an optional value creation lever but a standard toolkit component, though this creates challenges in forecasting actual impact and ROI when the future is uncertain. ## Technical and Organizational Challenges ### Talent Gap: The Dual-Hat Problem Nate identified a significant talent gap that isn't adequately addressed: organizations need people who can think about business value creation while also understanding how to effectively prompt and work with AI models. Pure data science PhDs aren't the answer, nor are pure business operators—the need is for dual-hat individuals who bridge both domains. Many people can put prompts into systems and generate code but have no idea what to do with it. Simultaneously, deep technologists often lack understanding of value creation in PE portfolio companies. This talent gap represents a more fundamental people infrastructure problem than many organizations realize. ### Communication Skills for LLM Effectiveness Chad made an insightful observation about a commonly overlooked enablement gap: organizations distribute tools expecting employees to use them effectively without training on the fundamental skill required—communication. The irony in IT organizations is that strong communicators aren't always abundant, yet large language models require clear communication of intent and context to produce valuable outputs. Senior developers and leaders who regularly work with junior developers and know how to communicate task requirements excel with LLMs. Junior developers and those without leadership experience struggle because they haven't developed these communication and task management skills. This applies across entire businesses, not just IT, yet organizations aren't investing in teaching people how to effectively communicate with AI systems. ### Data Security and Access Control An audience question about AI security highlighted that this remains an unsolved challenge across the industry. Warburg Pincus confirmed through recent discussions with portfolio CTOs that nobody has clear answers. Security manifests across multiple dimensions: usage policies for data (especially in regulated industries), quality issues when offering services like reports and chatbots, validation and guardrails for model outputs, telemetry for monitoring all activities, and ensuring proper test harnesses for model refreshes. An early challenge Chad highlighted was data aggregation risk: individually mundane pieces of data, when aggregated through AI queries, could reveal material insights that shouldn't be accessible to certain users—similar to why classified documents become unclassified, yet individual pieces seem innocuous. The defense-in-depth approach using tools like AWS Bedrock guardrails and agent core policies provides more granular control than previously available, but the problem isn't fully solved. Early implementations required careful data segregation to ensure AI tools could only access appropriate information and users could only explore certain knowledge paths. This remains an evolving area where solutions continue to develop. ### Cultural Resistance and Change Management The psychology of AI adoption emerged as critical. Robbie used an analogy of dropping a lion and zebra in the African wilderness—both face survival challenges but frame problems fundamentally differently. Mandating that teams use agentic approaches without new hires creates negative friction. Reframing the same constraint as "pretend you're a startup with a tiger team—see what you can accomplish with this small group" energizes people because it feels like green field innovation with rules removed. This psychological reframing, making people feel like they're in startup mode rather than under constraint, dramatically improved buy-in and enthusiasm. The broader organizational challenge involves avoiding what Chad called "transformation antibodies"—people who resist change and will point to failures to poison the well against AI initiatives. This makes early wins critical not just for demonstrating capability but for building momentum and keeping naysayers at bay. However, success also requires leadership that understands AI programs need time, perseverance, and acceptance of failures along the journey. ## Strategic Approaches and Recommendations ### Balancing Quick Wins with Bold Vision When sanctioning AI projects, the panel advocated a two-pronged approach. Quick wins are essential for building trust with stakeholders, establishing initiative momentum, developing organizational muscle memory, and securing continued investment. Showcasing achievable victories early prevents creating organizational cynicism if hyped projects fail. However, focusing exclusively on small ideas risks falling into ROI questioning where value seems insufficient. Therefore, organizations need parallel tracks: pursue quick wins for excitement and credibility while maintaining big, bold ideas broken into smaller milestones to demonstrate progress. Without focus on transformative opportunities, leadership will likely reprioritize resources within months. ### Avoiding Common Pitfalls Several anti-patterns emerged from the discussion. First, hesitate before launching big transformation projects in preparation for AI. Chad noted that companies would identify data challenges, step back to build comprehensive data governance, and spend all their time on data cleanup and governance—a business problem requiring all stakeholders, not just IT—while seeing no value because they're just establishing rigor. This slows progress to a crawl and creates cynicism about why the work is being done. Second, don't wave the AI "magic wand" at everything. Not all workflows benefit from AI, and some automation destroys valuable personalized service without justifiable ROI. Third, AI as a technology solution looking for problems doesn't work—start with business problems. ### Evolution Toward Agentic Systems While the panel focused primarily on current production realities, multiple speakers referenced the upcoming shift toward agentic AI. Robbie mentioned that code might not need to be aesthetically pleasing for human consumption if it's written for agentic consumption. Chad challenged an audience member's concern about architecture designs behind generated code by suggesting feeding architectural issues back into agents to automatically modify deployment approaches. The implication is that the first wave of LLM production deployment focused on augmentation and assistance, while the next wave will involve more autonomous agentic systems—a journey that's "just starting" despite three years since ChatGPT's launch. ## Regulatory and Industry-Specific Considerations Abrigo's position serving community banks under regulatory supervision provides important insights into LLM deployment in regulated industries. The company must carefully vet any new technology before internal employee rollout, let alone customer-facing deployment. This created initial friction around ChatGPT adoption in early 2023, requiring "conversation and debate" before proceeding. The regulatory context also shapes which use cases make sense: AI assistants providing immediate feedback on document verification can improve customer experience within compliance boundaries, while certain other automations might not be appropriate in banking workflows where auditability and human oversight remain critical. The philosophical approach distinguishes between deterministic and indeterministic workflows when considering AI application. Deterministic processes where outcomes must be predictable and auditable require different AI integration strategies than indeterministic interactions where AI can learn from exchanges and provide contextual assistance. Understanding this distinction prevents inappropriate AI deployment that could create regulatory risk or undermine necessary controls. ## Current State and Future Direction (2025) As of 2025, AI has shifted from optional to mandatory in private equity. Every investment committee presentation requires addressing AI risks and opportunities. The question has evolved from "might this be relevant?" to "defend why this won't be disrupted." Companies can no longer ignore AI in their strategic planning, competitive positioning, or operational roadmaps. However, significant gaps remain in ROI forecasting, security solutions, talent availability, and organizational readiness. The technology has matured enough that production deployments are common, but systematic approaches to maximizing value while managing risk continue evolving. The panel concluded by acknowledging that despite three years of progress, the industry is "just starting" to see things in production effectively. The impact of agents and other emerging capabilities haven't been fully explored. The journey will continue for a long time with much collective learning ahead. This tempered assessment, coming from practitioners with extensive production experience, provides important perspective against overhyped vendor claims about AI transformation being complete or straightforward.