## Overview Spotify, the global audio streaming platform, has developed an LLMOps infrastructure to deliver contextualized recommendations through personalized narratives. This case study explores how the company adapted open-source LLMs to enhance music discovery by generating explanations for recommendations and powering their AI DJ feature with real-time, culturally-aware commentary. The work represents a significant production deployment of LLMs in a high-scale consumer application serving millions of users. The core insight driving this initiative is that traditional recommendation systems present content through cover art and metadata, but users often lack the context to understand why something was recommended to them. By leveraging LLMs to generate personalized narratives—explanations that feel like a friend's recommendation—Spotify aims to increase user confidence and engagement with unfamiliar content. ## Backbone Model Selection and Criteria Spotify's approach to LLMOps begins with establishing a robust backbone model that can be adapted for multiple use cases. The company articulates specific criteria for backbone model selection that reflect practical production considerations: The backbone must possess broad world knowledge covering general and domain-specific information about music, podcasts, and audiobooks. This reduces the need for extensive retraining when crafting contextual recommendations. Functional versatility is equally important—the model should excel at diverse tasks including function calling, content understanding, topic extraction, and safety classification to enable rapid feature iteration. Community support emerged as a significant factor in their selection process. Strong open-source communities simplify fine-tuning workflows, provide efficient training and inference tools, and drive continuous improvements that help Spotify stay current with LLM advancements. Finally, AI safety is treated as a critical requirement, with backbone models needing built-in safeguards for handling sensitive content, preventing harmful outputs, and ensuring regulatory compliance. After evaluating multiple state-of-the-art models, Meta's Llama family emerged as their primary backbone for domain adaptation work. While the company maintains a portfolio of models across R&D teams, Llama's characteristics aligned well with their production requirements. ## Use Case 1: Recommendation Explanations The first production application generates concise explanations for music, podcast, and audiobook recommendations. Examples include phrases like "Dead Rabbitts' latest single is a metalcore adrenaline rush!" or "Relive U2's iconic 1993 Dublin concert with ZOO TV Live EP." These explanations aim to spark curiosity and enhance content discovery. The development process revealed several challenges that are common in production LLM deployments. Initial experiments with zero-shot and few-shot prompting of open-source models highlighted the need for careful alignment to ensure outputs are accurate, contextually relevant, and consistent with brand standards. Specific issues included artist attribution errors, tone inconsistencies, and factual inaccuracies (hallucinations). To address these challenges, Spotify implemented a human-in-the-loop approach. Expert editors created "golden examples" demonstrating proper contextualization and provided ongoing feedback to improve model outputs. This was combined with targeted prompt engineering, instruction tuning, and scenario-based adversarial testing to refine the generation quality. The results were compelling: online tests revealed that explanations containing meaningful details about artists or music led to significantly higher user engagement. In some cases, users were up to four times more likely to click on recommendations accompanied by explanations, with the effect being especially pronounced for niche content where users had less prior familiarity. ## Use Case 2: AI DJ Commentary Spotify's AI DJ, launched in 2023, represents a more complex production deployment requiring real-time, personalized commentary. The DJ serves as a personalized AI guide that understands listener music tastes and provides tailored song selections alongside insightful commentary about artists and tracks. A key challenge for LLM-based DJ commentary is achieving deep cultural understanding that aligns with diverse listener preferences. Music editors with genre expertise and cultural insight play a central role in this process. By equipping these editors with generative AI tools, Spotify scales their expertise while ensuring cultural relevance in model outputs. Through extensive comparisons of external and open-source models, the team found that fine-tuning smaller Llama models produces culturally-aware and engaging narratives on par with state-of-the-art alternatives, while significantly reducing costs and latency. This is a notable finding for LLMOps practitioners—smaller, domain-adapted models can match larger general-purpose models for specific tasks while offering better operational characteristics. Beta testing across select markets demonstrated that listeners who heard commentary alongside personal music recommendations were more willing to listen to songs they might otherwise skip, validating the approach. ## Domain Adaptation Infrastructure Spotify developed a comprehensive data curation and training ecosystem to enable rapid scaling of LLM adaptation. This infrastructure facilitates seamless integration of new models while enabling collaboration across multiple teams with expertise in dataset quality, task performance optimization, and responsible AI use. The training approaches employed include extended pre-training, supervised instruction fine-tuning, reinforcement learning from human feedback (RLHF), and direct preference optimization (DPO). Training datasets combine internal examples, content created by music domain experts, and synthetic data generated through extensive prompt engineering and zero-shot inference from state-of-the-art LLMs. Beyond the narrative generation use cases, Spotify evaluated LLMs ranging from 1B to 8B parameters across a growing set of Spotify-specific tasks, benchmarking zero-shot performance against existing non-generative, task-specific solutions. Llama 3.1 8B demonstrated competitive performance, leading to a multi-task adaptation targeting 10 Spotify-specific tasks. The multi-task adaptation approach aimed to boost task performance while preserving general model capabilities. The Massive Multitask Language Understanding (MMLU) benchmark served as a guardrail to ensure foundational capabilities remained intact. Results showed up to 14% improvement in Spotify-specific tasks compared to out-of-the-box Llama performance, with only minimal differences in MMLU scores from the zero-shot baseline. This demonstrates successful domain adaptation without catastrophic forgetting. ## Training Infrastructure and Resilience Distributed training is essential for the computational demands of billion-parameter models. The case study highlights a commonly overlooked aspect of production LLM training: resilience to system failures during lengthy training phases on multi-node, multi-GPU clusters. To address this, Spotify developed a high-throughput checkpointing pipeline that asynchronously saves model progress. By optimizing read/write throughput, they significantly reduced checkpointing time and maximized GPU utilization. This infrastructure consideration is critical for production LLMOps—training runs can take days or weeks, and robust checkpointing prevents costly re-starts from scratch. ## Inference and Serving Optimization The LLMOps challenges extend beyond training to efficient serving for both offline and online use cases. Spotify employs lightweight models combined with advanced optimization techniques including prompt caching and quantization to achieve efficient deployment. The goal is minimizing latency while maximizing throughput without sacrificing accuracy. Integration of vLLM, the popular open-source inference and serving engine, is described as a "game-changer" that delivered significant serving efficiencies and reduced the need for custom optimization techniques. vLLM enables low latency and high throughput during inference, allowing real-time generative AI solutions to reach millions of users. The flexible nature of vLLM also facilitated seamless integration of cutting-edge models like Llama 3.1, including the 405B variant, immediately upon release. This capability enables rapid benchmarking of new technologies and leveraging very large models for applications like synthetic data generation—even if these larger models aren't used directly in production serving. ## Critical Assessment While Spotify presents compelling results, several aspects warrant balanced consideration. The reported four-times improvement in click-through rates for recommendations with explanations is impressive but applies specifically to certain content types (especially niche content). The generalizability of these gains across all recommendation contexts is not fully established. The emphasis on open-source models like Llama reflects a strategic choice that offers cost advantages and operational control, but may require more internal expertise compared to using managed API services. The investment in custom training infrastructure, checkpointing pipelines, and serving optimization represents significant engineering overhead that may not be feasible for smaller organizations. The human-in-the-loop approach with expert editors, while effective for quality control, creates a potential bottleneck for scaling and may be costly to maintain. The balance between automated generation and human oversight is a recurring challenge in production LLM systems. That said, the case study provides valuable insights into production LLMOps practices: the importance of backbone model selection criteria, the effectiveness of domain adaptation over general-purpose prompting, the role of multi-task training in preserving general capabilities while improving specific tasks, and practical infrastructure considerations like checkpointing and inference optimization. The work demonstrates that smaller, well-tuned models can compete with larger alternatives for domain-specific tasks while offering better latency and cost characteristics—a finding with broad applicability across the industry.