## Overview LiftOff LLC conducted a comprehensive evaluation of self-hosting DeepSeek-R1 language models on AWS infrastructure as a potential replacement for commercial AI services. This case study provides valuable insights into the practical challenges and economic realities of deploying open-source large language models in production environments, particularly for startup-scale operations. The company's primary motivation was to explore whether they could reduce dependency on paid AI coding assistants while gaining greater control over their AI infrastructure. They were particularly interested in the privacy, security, and customization benefits that self-hosting could provide, along with potential long-term cost savings. ## Technical Implementation The deployment architecture centered around AWS EC2 GPU instances, specifically testing g5g.xlarge (4 vCPU, 8 GB RAM, 16 GB GPU) and g5g.2xlarge (8 vCPU, 16 GB RAM, 16 GB GPU) configurations. The team selected the G5 series for its balance of GPU performance and pricing, along with compatibility with CUDA and machine learning frameworks. Their deployment stack emphasized simplicity and rapid iteration. Docker served as the containerization platform, paired with NVIDIA Container Toolkit to ensure consistent environments with GPU access. This approach helped eliminate the common "it works on my machine" issues that often plague ML deployments. For model serving, they utilized Ollama, which simplified local model loading significantly, while OpenWeb UI provided a user-friendly, ChatGPT-like frontend interface suitable for internal testing by engineers. ## Model Performance Analysis The team conducted extensive benchmarking across four DeepSeek-R1 model variants: 1.5B, 7B, 8B, and 16B parameters. Their performance analysis revealed several critical insights about the trade-offs between model size, quality, and operational efficiency. The smallest model (1.5B parameters) demonstrated high efficiency in terms of tokens per second and operational cost but fell short of production-quality responses. The responses lacked the sophistication and accuracy required for their intended use cases, making it unsuitable as a replacement for commercial alternatives. Mid-tier models (7B and 8B parameters) showed noticeable improvements in response quality but still didn't meet the threshold needed to fully replace commercial LLMs. While these models performed better than the 1.5B variant, they introduced their own operational challenges, particularly around context handling and stability under load. The largest model tested (16B parameters) offered the best response quality but came with significant operational trade-offs. The team experienced slower throughput, dramatically higher costs, and severely limited concurrency capabilities. Performance degradation became particularly pronounced with longer contexts, with token generation speeds dropping below 30 tokens per second, which they identified as the threshold where responses begin to feel sluggish to users. ## Operational Challenges and Technical Hurdles The deployment process revealed several significant technical challenges that highlight the complexity of self-hosting large language models. GPU memory limitations emerged as a primary concern, with the 16B model routinely crashing under load until the team implemented aggressive quantization techniques and heavily reduced batch sizes. Even with these optimizations, inference stability remained unreliable, especially during peak throughput periods. Performance degradation with longer context windows posed another significant challenge. As prompt context increased, particularly with the 8B and 16B models, token generation speeds consistently dropped below their identified 30 tokens per second threshold. This led to noticeably slower response times and, in some cases, complete system crashes when the context window approached maximum limits. The complexity of performance tuning proved more demanding than anticipated. Despite the models being pretrained, achieving optimal throughput required significant engineering effort. Minor adjustments to batch size, sequence length, or context window parameters sometimes introduced unpredictable latency or runtime crashes, requiring extensive testing and optimization cycles. ## Cost Analysis and Economic Realities The economic analysis revealed stark differences between self-hosting and SaaS alternatives. A single g5g.2xlarge instance running 24/7 costs approximately $414 per month, while equivalent SaaS services like ChatGPT Plus cost $20 per user per month. For their target scale of under 100 internal users, the SaaS approach proved dramatically more cost-effective. Beyond the direct infrastructure costs, the team identified significant hidden operational expenses. These included setup time, DevOps overhead, ongoing maintenance, and the engineering effort required for optimization and troubleshooting. When factoring in these additional costs, the economic gap between self-hosting and SaaS solutions widened considerably. The cost analysis highlighted that self-hosting only begins to make economic sense at much larger scales or when specific requirements around data privacy, customization, or regulatory compliance justify the additional expense and complexity. ## Infrastructure Scaling Insights The experience provided the team with newfound appreciation for the engineering challenges faced by companies operating at the scale of OpenAI or DeepSeek's hosted infrastructure. Even supporting fewer than 100 internal users pushed the limits of what they could reliably run on GPU-backed EC2 instances with the 8B and 16B models. This realization underscored the massive infrastructure and optimization demands that grow exponentially with model size. The team recognized that delivering responsive, stable, and cost-effective LLM experiences at global scale requires sophisticated engineering capabilities that extend far beyond basic model deployment. ## Production Readiness Assessment The case study reveals several key considerations for production readiness of self-hosted LLMs. Memory management becomes critical with larger models, requiring careful attention to quantization strategies and batch processing optimization. The team learned that concurrent request handling must be optimized from the outset, as retrofitting concurrency capabilities proved challenging. Stability and reliability emerged as ongoing concerns, particularly with larger models under varying load conditions. The unpredictable nature of performance degradation with longer contexts suggests that production deployments would require sophisticated monitoring and failover mechanisms. ## Future Outlook and Recommendations Despite the current challenges, the team maintains optimism about the future of self-hosted LLMs. They note improving GPU access and the emergence of specialized hardware like Nvidia DIGITS, which could enable developers to prototype, fine-tune, and inference reasoning AI models locally more effectively. Their recommendation for startups and growing enterprises is clear: at smaller scales, SaaS LLM offerings currently deliver superior value for money. However, they advocate for continued experimentation and preparation for future opportunities as the technology landscape evolves. ## Broader Implications for LLMOps This case study illustrates several important principles for LLMOps practitioners. The importance of comprehensive cost-benefit analysis extends beyond direct infrastructure costs to include operational overhead and engineering time. Performance benchmarking must consider not just raw throughput but also user experience thresholds and real-world usage patterns. The experience also highlights the value of starting with simpler deployment approaches and gradually scaling complexity as requirements and capabilities mature. The team's choice of Docker, Ollama, and OpenWeb UI created a foundation that could be incrementally improved rather than requiring wholesale architectural changes. Finally, the case study demonstrates the importance of realistic expectations when evaluating emerging technologies. While self-hosting open-source LLMs offers compelling theoretical advantages, the practical implementation challenges and economic realities must be carefully weighed against organizational capabilities and requirements.