# Samsung's Autonomous Fab: LLMs and Multimodal AI in Semiconductor Manufacturing ## Overview and Vision This case study presents Samsung's ambitious effort to achieve a "fully autonomous fab" (fabrication facility) for semiconductor manufacturing. The speaker, a Samsung engineer, clarifies that this vision is not about replacing engineers but rather empowering them—drawing an analogy to autonomous driving where humans remain important but are freed from manual, repetitive tasks. The core framework for autonomous fab operations consists of three components: sensing (observing equipment behavior), analysis (understanding what's happening), and control (taking action based on insights). The semiconductor industry presents unique challenges for AI and LLM deployment. Modern chip fabrication, particularly at advanced nodes like 5nm, involves close to 1,000 process steps. Achieving the 80%+ yield required for profitability at these nodes (5nm, 4nm, 3nm) has become increasingly difficult compared to older technologies like 28nm or 14nm. This complexity creates an ideal use case for generative AI and LLM-based solutions. ## The Production Environment and Data Landscape Samsung's semiconductor manufacturing environment generates massive amounts of sensor data. The primary data type discussed is "trace data"—time-series readings captured every second from equipment sensors. For example, during an etching process, sensors monitor reflected power and forward power as plasma is initiated, maintained, and terminated. Even a simple two-step etch process generates complex signals that are difficult to monitor manually. The conventional approach involves capturing snapshots of specific process steps and computing summary statistics (average, standard deviation, min, max, initial slope, stability). More sophisticated methods involve windowing—observing only specific time periods rather than full steps. However, all these approaches historically required extensive manual effort and deep domain expertise to configure properly. Beyond trace data, the production environment includes: - Event logs documenting equipment alarms and abnormal conditions - Maintenance histories and component information - Knowledge bases containing engineering notes and troubleshooting guides - Metrology and inspection data including wafer images - Optical Emission Spectroscopy (OES) data showing plasma characteristics ## LLM-Based Production Systems Already Deployed Samsung has already deployed LLM-based systems in production environments. One notable example is an engineering assistant that helps technicians diagnose equipment issues. When an engineer encounters an error, they can prompt the system with questions like "I got error ABC, what should I do?" The system responds with historical context about similar errors and provides actionable suggestions based on past resolutions. This system is described as truly multimodal, utilizing images, text, and graphs. The speaker emphasizes this is "not the future—we already have this kind of service in place." The system leverages RAG (Retrieval-Augmented Generation) architecture to ground LLM responses in actual historical data and documentation. However, the speaker candidly acknowledges ongoing challenges with hallucination and incorrect answers, noting that they are "suffering from the hallucination" but continue working to "make the best out of it utilizing RAG." ## Knowledge Graph Integration for Sensor Alignment A significant production challenge in semiconductor manufacturing involves sensor naming inconsistencies across equipment. Different chambers may have the same type of sensor but with different naming conventions. From an equipment engineer's perspective, they care about whether "Chamber A, Sensor A" behaves correctly compared to "Chamber B, Sensor A" (named differently). From a wafer perspective, you need to understand that these are essentially the same sensor type performing similar functions. Previously, this required strict naming discipline enforced through training and penalties. Even with 99% accuracy, given hundreds of chambers and thousands of steps, errors accumulated significantly. Samsung now uses knowledge graphs combined with LLMs to automatically align and reconcile sensor naming, handling the inevitable human errors gracefully. A concrete example demonstrated the power of this approach: when equipment signals suddenly changed after maintenance, engineers couldn't identify any obvious differences through conventional analysis. Using knowledge graph community detection, Samsung identified that the network relationships between sensors had changed subtly. This led to discovering that a membrane outside the reaction chamber had slightly altered values—something no one had previously considered examining. ## Time Series Foundation Models: Current Limitations Samsung is actively evaluating time series foundation models for predicting equipment behavior. They've tested multiple models including Patch TST (from IBM), TimeLLM, and TimeGPT, applying them to their trace data. The results reveal significant challenges for production deployment: - Zero-shot performance was described as "very bad—no way close" - Pre-training improved predictions to "decent" levels - Fine-tuning and full fine-tuning produced visually reasonable predictions with low MAE However, the speaker was explicit that these models are not yet production-ready for control applications. Two major obstacles persist: - **Flat/stationary behavior**: Models struggle with stationary signal segments, producing predictions that look similar but aren't accurate enough - **Time delay/lag**: Predictions consistently lag behind actual signals by several seconds/data points, particularly during rapid transitions Most critically, for process control, Samsung needs accurate predictions at transition points and specific windows. The correlation R-squared values max out at 0.8 but are typically below 0.5 in the critical regions—insufficient for reliable process control. Samsung is collaborating with IBM, AI-tomatik, and other companies to improve these models, potentially providing semiconductor-specific training data to help foundation model developers tune their systems for microsecond-level industrial time series data. ## Multimodal Applications: OES and Defect Detection ### Optical Emission Spectroscopy (OES) Analysis OES generates three-dimensional data: wavelength (x-axis), time (y-axis), and intensity (z-axis). This shows plasma composition changes during processing—different chemical peaks (N2, O2, etc.) indicate what's happening inside the chamber. Conventional endpoint detection (EPD) requires pre-determining which specific wavelength peaks to monitor. With generative AI, Samsung can process the entire 3D OES data and make decisions dynamically, rather than relying on predetermined peaks. This is significant because optimal indicator peaks may vary during processing, and traditional approaches cannot adapt mid-process. The system also helps with peak identification. While humans can visually spot peaks, automated algorithms require constant updating and aren't always accurate. In one case, AI detected very small peaks next to expected peaks that turned out to indicate a gas leak—a subtle contamination that would have been ignored manually but was caught through AI analysis, preventing further losses. ### Defect Detection and Root Cause Analysis For wafer inspection, the system processes wafer map images showing suspected defects (marked as red dots on the wafer surface). Traditionally, determining defect types required physically zooming into individual defects—a time-consuming process that could only cover a sample of potentially thousands of defects per wafer. Samsung's multimodal approach enables classification based on the wafer-level pattern distribution, potentially saving significant time and resources. For yield analysis, engineers can prompt the system with questions about error patterns, asking whether similar issues occurred previously and what the root causes were. ## Reinforcement Learning for Process Control Samsung has implemented reinforcement learning for equipment control, specifically for RF (radio frequency) matching in plasma processes. Traditional matching systems only use information from outside the reaction chamber, but Samsung's RL approach incorporates sensor data from inside the chamber. Results showed dramatic improvement: conventional matching required many iterations to achieve optimal impedance matching, while RL-based matching achieved results in just 5-6 iterations. The speaker expressed enthusiasm about applying this approach to broader process control beyond just RF matching. ## Physics-Informed Neural Networks Given the challenge of limited training data (few wafers but many input variables), Samsung employs Physics-Informed Neural Networks (PINNs). These incorporate known physical relationships to reduce the effective dimensionality of the learning problem and improve signal-to-noise ratios. This approach helps predict critical parameters like circuit width (CD), profile characteristics, and defect rates. ## Data Standardization Challenges A significant operational challenge is data format inconsistency across equipment vendors and fabs. The speaker emphasized that without industry-wide standards, each fab (Samsung, TSMC, Intel) develops proprietary data formats, preventing model transferability and collaboration. Samsung is leading a SEMI industry task force on "Equipment Data Publication" to define standardized data formats. They're also building private cloud infrastructure where authorized partners can access data for analytics and model development while maintaining data security. ## Collaboration and Future Directions The speaker mentioned collaboration with multiple organizations: - IBM (time series foundation models, Patch TST) - AI-tomatik (AI automation) - NVIDIA, Exus, and other industry partners - AI Alliance (foundation model working group, data catalog initiatives) - SEMI industry task forces for standardization The discussion revealed that the semiconductor industry's microsecond-level time series data represents a novel challenge for existing foundation models, which were typically trained on different temporal scales. This gap between available pre-trained models and industrial requirements highlights the need for domain-specific model development and fine-tuning. ## Key Takeaways for LLMOps Practitioners This case study illustrates several important LLMOps lessons: - Production LLM systems can deliver value even with acknowledged limitations (hallucinations) - RAG is essential for grounding responses in domain-specific historical data - Knowledge graphs complement LLMs for entity resolution and relationship analysis - Time series foundation models require significant domain adaptation before production use - Multimodal approaches are necessary for complex industrial environments - Reinforcement learning offers advantages for control applications over traditional deep learning - Data standardization remains a major barrier to model portability across organizations