## Overview This case study comes from a PTC Talks webinar featuring Jonathan Tipper (Generative AI Lead for Industry X at Accenture) and Jan Nicholls (EMEA Manufacturing Practice Lead at Accenture). The session, held in early 2024, provides insights into how Accenture has been approaching generative AI for manufacturing clients through their Industry X practice—a 26,000-person global team focused on digitizing engineering and manufacturing operations. The presentation represents a consulting perspective on LLMOps for manufacturing, showcasing validated use cases developed through rapid experimentation rather than a single production deployment. While the content is inherently promotional for Accenture's services, it offers valuable insights into real-world challenges and lessons learned when deploying LLMs in industrial contexts. ## Context and Approach Accenture's Industry X practice has been applying generative AI to manufacturing since ChatGPT's emergence in late 2022. Their approach follows a three-phase methodology: Explore, Experiment, and Execute. The presenters emphasized the "Experiment" phase as particularly critical for manufacturing, arguing that unlike creative industries where generative AI applications may be more obvious, manufacturing use cases require rigorous validation before scaling. The team conducts experiments in four to six week cycles at relatively low investment, measuring impact not just on direct processes but also on people, data, and architecture. They reference Harvard research on the "technological frontier"—the concept that AI excels at some difficult tasks while failing at seemingly easy ones, making upfront experimentation essential. ## Validated Use Cases The presentation covered nine validated use cases that emerged from their experimentation program, categorized into three capability areas: ### Content Generation The first category focuses on generative AI's core strength: creating new content. Two prominent examples include test case generation for software-defined vehicles and technical publication generation for automotive. In the test case generation scenario, the system takes requirements (in text or image prototype form), generates test cases, and then produces test scripts. For technical publications, the solution ingests vehicle features and details, applies proper structure and formatting, and generates documentation. Both applications demonstrated 40-50% reductions in manual effort, freeing workers for higher-value activities. ### Data Interaction and Insights The second category addresses a persistent challenge in manufacturing: getting actionable insights from industrial data. Since manufacturing data typically isn't in the public domain and thus wasn't part of LLM training data, Accenture developed approaches to make this proprietary data accessible through conversational interfaces. One example involves dynamic digital dashboards that respond to natural language queries about KPIs, OEE metrics, and trends. Another integrates a copilot assistant into Accenture's operations twin (digital twin for operations), enabling users to perform root cause analysis and identify trends by "talking to your factory." These applications leverage retrieval and contextual integration rather than relying solely on the LLM's parametric knowledge. ### Workflow-Integrated Copilots The third category emphasizes human augmentation through copilots embedded directly into existing tools and workflows. Examples include integration with standard operating procedures, log books, model-based systems engineering (MBSE) tools, and PTC Codebeamer for requirements quality checking. The Codebeamer integration performs requirements quality checks natively within the tool, applying ISO standards and compliance requirements through generative AI. The virtual mentor for manufacturing represents their most adoption-focused solution, providing shop floor operators with a copilot that synthesizes standard operating procedures, log books, and tribal knowledge on the fly based on their current position in the assembly process. ## Technical Architecture Considerations ### Model Selection and Multi-Agent Systems The presenters noted that they don't select a single platform or model provider. Instead, they evaluate different models during each experiment based on client needs and use case requirements. They mentioned OpenAI, Meta's open-source offerings, and hyperscaler platforms (Microsoft, Amazon, Google) as common options. A key theme was the shift toward "multi-modal, multi-agent solutions." Jonathan Tipper characterized ChatGPT as "so 2022," predicting that 2024 would see significant growth in agent-based architectures. These agents function as "specialized virtual team members" that perform tasks, coordinate with each other, and return results to human users. This architectural pattern enables tackling complex problems that simple chatbots cannot address. ### Offline and Local Deployments A significant production consideration for manufacturing is network connectivity—operators often work in environments without internet access. Accenture confirmed they have developed offline, local deployment options, either in customer data centers or directly on-machine. These deployments rely on open-source, deployable models rather than cloud-based APIs, with different architectural patterns to accommodate air-gapped environments while maintaining data integrity and system-level integration. This capability is particularly relevant for aerospace, defense, and other sectors with strict data sovereignty requirements, where cloud model access may not be acceptable. ## Lessons Learned and LLMOps Best Practices The presentation outlined four key learnings from their experimentation program: ### Start with Your Own Data This includes not just industrial sensor data but also processes, workflows, standards, operating procedures, log books, and tribal/expert knowledge. The emphasis on "tribal knowledge trapped within expert individuals" suggests significant effort goes into knowledge capture and structuring before LLM integration can succeed. ### Avoid the Knowledge Management Trap While it's tempting to deploy a general-purpose enterprise knowledge management chatbot, the presenters explicitly advised against this approach. LLMs are "not natively good" at solving broad knowledge management challenges without substantial surrounding infrastructure. Instead, they recommend focusing on specific use cases that radically augment existing high-value activities, driving adoption through workflow integration rather than general-purpose search. ### Embrace Multi-Agent Orchestration Rather than single chatbots, the architecture should support multiple specialized agents coordinating to solve complex problems. This requires thinking bigger about the problems being tackled and designing systems that can grow in capability over time. ### Maintain Human-in-the-Loop All demonstrated solutions involve human oversight and collaboration. Test case generation, technical publication creation, and copilot interactions all position the AI as an augmentation tool rather than a fully automated system. The presenters stated explicitly: "These aren't fully automated running and unsupervised—not yet, not this year." Trust-building through human verification is essential for adoption. ## Risk Management and Responsible AI The presentation acknowledged substantial risks with generative AI: unreliable results, confidence calibration issues, model bias, liability questions, compliance concerns, and workforce impact. Accenture references their "Responsible AI" program, which they've operated since 2016, as a framework for addressing these concerns. In manufacturing contexts, where outputs may affect safety-critical systems or regulatory compliance, these considerations are particularly important. ## Results and Value Claims The reported value metrics vary significantly by use case: - 40-50% effort reduction for technical publication and test case generation - Up to 80% reduction in time for converting functional requirements into MBSE models - Various efficiency improvements in inspection, maintenance planning, and cycle time The presenters appropriately cautioned that value depends heavily on industry, current maturity level, data foundation, and existing automation. They explicitly recommend honest measurement during experimentation rather than assuming value will materialize. ## Failed Experiments and Honest Assessment When asked about failed experiments, Jonathan Tipper mentioned engineering drawing interpretation as an example where early 2023 capabilities were insufficient. While models could process vector-based drawings to some degree, they couldn't match the expertise of experienced human engineers. He noted that many experiments produced "cool demos" that weren't actually valuable enough for daily workflow adoption—an important distinction for production deployments where adoption is the ultimate success metric. ## Limitations and Considerations This case study represents a consulting firm's perspective on generative AI for manufacturing. While the experiences described appear genuine, the presentation is inherently promotional, and specific client names or detailed implementation architectures aren't disclosed. The metrics cited are from experiments rather than long-running production systems, and the four-to-six-week experimentation cycles, while practical for rapid learning, may not capture challenges that emerge only at scale or over extended periods. The emphasis on rapid technological change—with acknowledgment that answers would differ "in three months time"—also suggests that some recommended approaches may have already evolved since the January 2024 presentation.