## Overview This presentation, delivered at a technology conference, provides a candid examination of three enterprise GenAI implementations and the journey from pilot to profit. The speaker, representing what appears to be a consulting or implementation firm, shares lessons learned from deploying LLM-based solutions across automotive manufacturing, in-flight entertainment systems, and telecommunications. The talk is notable for its honest assessment of challenges, failures, and the realistic economics of enterprise AI deployments, offering practical insights that contrast with more optimistic industry narratives. The speaker builds on a previous year's presentation about lessons learned in enterprise GenAI adoption, now focusing on whether these implementations actually generated revenue. The central thesis is that reaching pilot success is only the beginning—the real challenge lies in achieving "pilot to profit" transformation. ## Case Study 1: Automotive Manufacturing Equipment Maintenance The first case involves a top-10 global automaker facing a critical workforce challenge. The manufacturing line relied on an aging population of maintenance experts responsible for keeping equipment running—from chassis assembly to paint and finishing. The business criticality was severe: any machine downtime stops the entire production line, directly impacting revenue generation. The initial problem seemed straightforward: build a RAG-based system that could learn from service manuals and help new, less experienced employees maintain machinery. The team loaded all available service manuals into the system and built what they considered a successful pilot. The answers were not only accurate but synthesized information across multiple manuals to provide comprehensive responses. However, upon deployment, the same experts who had provided the service manuals rejected the system, leading to what the speaker calls "pilot paralysis"—a state where teams are happy with their pilot but cannot progress beyond it. After six months of operation, the ROI analysis showed that even with a 36-month projection, the system would not deliver acceptable returns. The root cause investigation revealed a critical insight: while the system's answers were "theoretically fine," the maintenance experts had developed faster, practical solutions through years of experience. This institutional knowledge was not documented in the service manuals but existed in 10,000 incident reports—informal, non-standardized documents where workers noted quick fixes when equipment failed. The solution involved implementing a multi-agent architecture with two distinct agents: a "theoretical expert" trained on service manuals and a "practical expert" trained on incident reports. These agents would evaluate solutions and recommend the most appropriate action given time constraints, with the system tracking outcomes for continuous improvement. This architectural change increased initial costs significantly due to the data preparation required for unstructured, non-standardized incident reports. Maintenance costs rose slightly due to increased token sizes. However, the system began delivering the promised savings, achieving break-even at approximately 14 months. The key insight here was identifying "adoption triggers"—the hidden lever that determines whether users will actually embrace the system. Two critical triggers emerged: getting expert content into the system and connecting directly to the incident reporting system so workers didn't have to change their behavior to contribute data. ## Case Study 2: In-Flight Entertainment Device Predictive Maintenance The second case involves a $25 billion company that manufactures and maintains in-flight entertainment devices for nearly all global airlines. They had been identified as one of the 50 most innovative companies, yet faced complaints about device failure rates affecting their business relationships. The technical opportunity centered on the massive data transfer that occurs when planes land—between 800GB to 1TB of data including sensor readings and entertainment device logs capturing every click, volume change, selection, and reboot. The initial approach was to process these logs through a GenAI system to detect deviation patterns and predict failures before they occurred, enabling proactive device swaps during routine maintenance. The pilot covered approximately 20 aircraft types and successfully processed logs to detect patterns. However, the finance team rejected the approach because it wouldn't scale. The maintenance costs were too high because the system processed every log file for every seat on every aircraft for every flight—when scaling to the 10,000+ daily flights operated by major carriers, this became economically unsustainable. The solution required stepping back and implementing a multi-tiered architecture. The team integrated traditional log analysis tools (Splunk, ELK, or similar) to first perform time-series analysis and detect patterns of deviation. Only the specific logs from identified problematic devices on specific dates would then be passed to the GenAI system for root cause analysis—determining whether issues were device problems, wiring problems, or aircraft issues. This approach dramatically reduced maintenance costs by eliminating "token bloat." Over time, as the system identified recurring patterns, these could be codified directly into the log analysis layer, further reducing the need for GenAI processing. Eventually, only 5-7% of logs required GenAI analysis. The hidden lever here was "lean workflows"—using GenAI only where necessary and leveraging classical AI and traditional tools for appropriate tasks. The speaker candidly admits this seems obvious in retrospect but notes the team was initially "excited about doing this and passing everything through the Gen tool just because it could do it." ## Case Study 3: Telecommunications Customer Engagement The third case involves an Asian telecommunications company with approximately 40% market share, targeting small and medium businesses. Despite recent 50% year-over-year EBIT growth, they struggled to convert customer interest into sales. Their customers—like someone opening a new coffee shop—didn't know what products they needed or how services related to each other. The initial solution was a lightweight chatbot that could navigate customers through their service catalog and help them identify appropriate bundles. The system was deployed for one region and appeared to function as designed—even generating service combinations that salespeople hadn't considered. The ROI analysis was devastating: despite approximately $400K in initial costs, the savings were negligible and every dollar invested appeared unrecoverable. Investigation revealed that customers were using the chatbot to understand their needs, then purchasing from competitors. The company was inadvertently educating the market for their competition and losing market share. The rapid solution involved deploying a parallel agent that would automatically configure deployable solutions during conversations. Using infrastructure-as-code approaches (mentioned as "similar to Terraform templates"), the system would build ready-to-deploy configurations in real-time. When customers expressed interest in POS systems, CRM tools, or network services, the backend would configure the solution. At conversation's end, customers could deploy immediately, capturing the sale before they left the site. For services requiring physical installation, the system captured scheduling commitments—locking customers into specific appointment times. This quote-to-cash acceleration transformed the economics. The implementation identified "revenue accelerators" as the hidden lever, encompassing three strategies: capturing sales immediately, enabling dynamic upselling and cross-selling during conversations (suggesting additional bandwidth for coffee shops expecting customers using WiFi, then security services to protect expanded networks), and offering dynamic discounts when customers hesitated. ## The Failed Case: HR Chatbot The speaker briefly mentions a fourth project that failed—an HR team wanted to automate responses to repetitive employee questions. The system was built with low temperature settings to provide conservative responses. However, usage logs revealed employees only wanted to ask about career paths and promotions, not the HR information the system was designed to provide. The speaker notes this is a case where saying "no" upfront would have been appropriate. ## Economics and Practical Guidance The speaker provides unusually transparent guidance on GenAI project economics: - **Initial investment range**: $200K to $1M for pilots - **Timeline**: 3-6 months minimum for pilot development - **ROI timeline**: 15-18 months is realistic; the speaker explicitly challenges Gartner's predictions of sub-12-month ROI, stating "I haven't seen ROI in less than a year" - **Team composition**: Requires prompt engineers, infrastructure specialists, DevOps engineers, and TPMs who can embed solutions into existing business processes ## Key Operational Principles The speaker concludes with principles for LLMOps success: **Patience**: Enterprise GenAI transformations take longer than anticipated, and teams must set realistic expectations with stakeholders. **Empathy for users**: Employees fear AI systems will replace them, and this concern must be acknowledged even when reassurances are provided. **Empathy for operators**: The rapidly evolving GenAI landscape means operations teams face constant pressure to update libraries, apply patches, and evaluate new tools. **Empathy for investors**: Leaders must justify significant investments to boards and need confidence-building data to support continued funding. The three hidden levers—adoption triggers, lean workflows, and revenue accelerators—represent the core operational insights for moving from successful pilots to profitable production deployments. The speaker emphasizes that identifying and pulling these levers early can significantly reduce time-to-value compared to discovering them through trial and error as their team did.