Tag
llm
44 posts with this tag
- The Anatomy of a Production Coding AgentA production coding agent isn't a prompt and a while loop. It's eight stages, each with different failure modes, costs, and human touchpoints. Here's the full pattern.kitaruagentsproductionHamza Tahir2026
- RLMs in Production: What Happens After the NotebookagentsproductionLLMOpsAlex Strick van Linschoten2026
- 8 Best DeepEval Alternatives: Which LLM Evaluation Framework is Better?In this article, you will learn about the best DeepEval alternatives that you can use for LLM evaluation.discoveryLLMOpsevaluation+1Hamza Tahir2025
- 9 Best Langfuse Alternatives to Trace, Evaluate, and Manage Prompts for Your LLM ApplicationIn this article, you learn about the best Langfuse alternatives for tracing, eval, prompt management, and metrics for LLM apps.discoveryLLMOpsevaluationHamza Tahir2025
- Smolagents vs LangGraph: Which One’s Easier to Build and Run AI AgentsIn this Smolagents vs LangGraph, we explain the difference between the two and conclude which one is the best to build AI agents.LLMOpsframeworkmlops+1Hamza Tahir2025
- What are the 9 Best LLM Observability Tools Currently on the Market?Compare the 9 best LLM observability tools for agent tracing, evaluation, cost tracking, and production debugging.LLMOpsagentsframework+1Hamza Tahir2025
- We Tried and Tested 8 Langflow Alternatives for Production-Ready AI WorkflowsDiscover the top 8 Langflow alternatives you can leverage to build and deploy AI agents.LLMOpsframeworkagents+1Hamza Tahir2025
- 7 Best Agentic AI Frameworks to Build Smarter AI WorkflowsDiscover the 7 best Agentic AI frameworks to help you build smarter AI workflows this year.agentsLLMOpsframework+1Hamza Tahir2025
- Production-Ready AI Agents: Why Your MLOps Stack is the Missing PieceLLMOpsapplied-zenmlagents+1Alex Strick van Linschoten2025
- LlamaIndex Pricing Guide: Everything You Must Know Before InvestingIn this LlamaIndex pricing guide, we discuss the costs, features, and value LlamaIndex provides to help you decide if it’s the right investment for your business.LLMOpsagentsframework+1Hamza Tahir2025
- CrewAI Pricing Guide: Plans and Features the Framework OffersIn this CrewAI pricing guide, we discuss the costs, features, and value CrewAI provides to help you decide if it’s the right investment for your business.LLMOpsgenaimultimodal+1Hamza Tahir2025
- Salesforce Agentforce Pricing Guide: How Much Does It Cost?In this Agentforce pricing guide, we discuss the costs, features, and value Agentforce provides to help you decide if it’s the right investment for your business.LLMOpsagentsframework+1Hamza Tahir2025
- The Agent Deployment Gap: Why Your LLM Loop Isn't Production-Ready (And What to Do About It)Comprehensive analysis of why simple AI agent prototypes fail in production deployment, revealing the hidden complexities teams face when scaling from demos to enterprise-ready systems.deploymentagentsLLMOps+1Alex Strick van Linschoten2025
- The Annotated Guide to the Maven Evals Course (by way of the LLMOps Database)Lessons from the Maven Evals course are combined with 50+ real-world case studies from ZenML's LLMOps Database to show how companies like Discord, GitHub, and Coursera implement the Three Gulfs model and Analyze-Measure-Improve lifecycle to transform failing LLM systems into production-ready applications.evaluationgenaillmops-databaseAlex Strick van Linschoten2025
- LlamaIndex vs LangGraph: How are They Different?In this LlamaIndex vs LangGraph article, we explain the differences between these platforms and when to use each one for optimal results.LLMOpsagentsautomation+1Hamza Tahir2025
- LangGraph vs CrewAI: Let’s Learn About the DifferencesLangGraph vs CrewAI compared on workflows, human-in-the-loop, security and current pricing, plus how to evaluate either agent's traces before you ship.LLMOpsorchestratorsagents+1Hamza Tahir2025
- Steerable Deep Research: Building Production-Ready Agentic Workflows with Controlled AutonomyLearn how to build production-ready agentic AI workflows that combine powerful research capabilities with enterprise-grade observability, reproducibility, and cost control using ZenML's structured approach to controlled autonomy.zenml-projectagentsapplied-zenmlAlex Strick van Linschoten2025
- Chat With Your ML Pipelines: Introducing the ZenML MCP ServerDiscover the new ZenML MCP Server that brings conversational AI to ML pipelines. Learn how this implementation of the Model Context Protocol allows natural language interaction with your infrastructure, enabling query capabilities, pipeline analytics, and run management through simple conversation. Explore current features, engineering decisions, and future roadmap for this timely addition to the rapidly evolving MCP ecosystem.LLMOpsmlopsmcp+1Alex Strick van Linschoten2025
- Multimodal LLM Pipelines: From Data Ingestion to Real-Time InferenceLearn how to build, fine-tune, and deploy multimodal LLMs using ZenML. Explore LLMOps best practices for deployment, real-time inference and model management.zenmlmultimodalzenml-project+1 Haziqa Sajid2025
- Making ML Documentation AI-Friendly: ZenML's Implementation of llms.txtDiscover how ZenML implements the llms.txt standard to make ML documentation more accessible to both AI assistants and humans. Learn about our modular approach using specialized documentation files, practical integration with AI development tools, and how this structured format enhances the developer experience across different context window sizes.documentationsummarizationJayesh Sharma2025
- LLMOps in Production: 457 Case Studies of What Actually WorksA comprehensive overview of lessons learned from the world's largest database of LLMOps case studies (457 entries as of January 2025), examining how companies implement and deploy LLMs in production. Through nine thematic blog posts covering everything from RAG implementations to security concerns, this article synthesizes key patterns and anti-patterns in production GenAI deployments, offering practical insights for technical teams building LLM-powered applications.LLMOpsllmops-databaseproduction+2Alex Strick van Linschoten2025
- Production LLM Security: Real-world Strategies from Industry Leaders 🔐Learn how leading companies like Dropbox, NVIDIA, and Slack tackle LLM security in production. This comprehensive guide covers practical strategies for preventing prompt injection, securing RAG systems, and implementing multi-layered defenses, based on real-world case studies from the LLMOps database. Discover battle-tested approaches to input validation, data privacy, and monitoring for building secure AI applications.securityLLMOpsllmops-database+1Alex Strick van Linschoten2025
- The Evaluation Playbook: Making LLMs Production-ReadyA comprehensive exploration of real-world lessons in LLM evaluation and quality assurance, examining how industry leaders tackle the challenges of assessing language models in production. Through diverse case studies, the post covers the transition from traditional ML evaluation, establishing clear metrics, combining automated and human evaluation strategies, and implementing continuous improvement cycles to ensure reliable LLM applications at scale.LLMOpsllmops-databasewandb+1Alex Strick van Linschoten2024
- Prompt Engineering & Management in Production: Practical Lessons from the LLMOps DatabasePractical lessons on prompt engineering in production settings, drawn from real LLMOps case studies. It covers key aspects like designing structured prompts (demonstrated by Canva's incident review system), implementing iterative refinement processes (shown by Fiddler's documentation chatbot), optimizing prompts for scale and efficiency (exemplified by Assembled's test generation system), and building robust management infrastructure (as seen in Weights & Biases' versioning setup). Throughout these examples, the focus remains on systematic improvement through testing, human feedback, and error analysis, while balancing performance with operational costs and complexity.LLMOpsllmops-databaseRAG+2Alex Strick van Linschoten2024
- Building LLM Applications that Know What They're Talking About 🔓🧠Explore real-world applications of Retrieval Augmented Generation (RAG) through case studies from leading companies in the ZenML LLMOps Database. Learn how RAG enhances LLM applications with external knowledge sources, examining implementation strategies, challenges, and best practices for building more accurate and informed AI systems.RAGLLMOpsllmops-database+1Alex Strick van Linschoten2024
- Demystifying LLMOps: A Practical Database of Real-World Generative AI ImplementationsThe LLMOps Database offers a curated collection of 300+ real-world generative AI implementations, providing technical teams with practical insights into successful LLM deployments. This searchable resource includes detailed case studies, architectural decisions, and AI-generated summaries of technical presentations to help bridge the gap between demos and production systems.best-practiceseducationLLMOps+3Alex Strick van Linschoten2024
- LLMOps Lessons Learned: Navigating the Wild West of Production LLMs 🚀Explore key insights and patterns from 300+ real-world LLM deployments, revealing how companies are successfully implementing AI in production. This comprehensive analysis covers agent architectures, deployment strategies, data infrastructure, and technical challenges, drawing from ZenML's LLMOps Database to highlight practical solutions in areas like RAG, fine-tuning, cost optimization, and evaluation frameworks.best-practicesgenaiLLMOps+2Alex Strick van Linschoten2024
- Everything you ever wanted to know about LLMOps Maturity ModelsAs organizations rush to adopt generative AI, several major tech companies have proposed maturity models to guide this journey. While these frameworks offer useful vocabulary for discussing organizational progress, they should be viewed as descriptive rather than prescriptive guides. Rather than rigidly following these models, organizations are better served by focusing on solving real problems while maintaining strong engineering practices, building on proven DevOps and MLOps principles while adapting to the unique challenges of GenAI implementation.AzureAzureMLbest-practices+6Alex Strick van Linschoten2024
- LLM Evaluation & Prompt Tracking Showdown: A Comprehensive Comparison of Industry ToolsAs Large Language Models (LLMs) revolutionize software development, the challenge of ensuring their reliable performance becomes increasingly crucial. This comprehensive guide explores the landscape of LLM evaluation, from specialized platforms like Langfuse and LangSmith to cloud provider solutions from AWS, Google Cloud, and Azure. Learn how to implement effective evaluation strategies, automate testing pipelines, and choose the right tools for your specific needs. Whether you're just starting with manual evaluations or ready to build sophisticated automated pipelines, discover how to gain confidence in your LLM applications through robust evaluation practices.LLMOpsJayesh Sharma2024
- The State of LLM Operations or LLMOps: Why Everything is Hard (And That's OK)Machine Learning (ML) adoption is gaining momentum, but challenges include robust pipelines, quality issues, and scale monitoring. Recognizing and overcoming these challenges is crucial.Thought LeadershipmlopsLLMOpsAlex Strick van Linschoten2024
- Automating Lightning Studio ML Pipelines For Fine Tuning LLM (s)In the AI world, fine-tuning Large Language Models (LLMs) for specific tasks is becoming a critical competitive advantage. Combining Lightning AI Studios with ZenML can streamline and automate the LLM fine-tuning process, enabling rapid iteration and deployment of task-specific models. This approach allows for the creation and serving of multiple fine-tuned variants of a model, with minimal computational resources. However, scaling the process requires resource management, data preparation, hyperparameter optimization, version control, deployment and serving, and cost management. This blog post explores the growing complexity of LLM fine-tuning at scale and introduces a solution that combines the flexibility of Lightning Studios with the automation capabilities of ZenML.lightning aiHamza Tahir2024
- Using ZenML+ Databricks to Supercharge LLM DevelopmentThe integration of ZenML and Databricks streamlines LLM development and deployment processes, offering scalability, reproducibility, efficiency, collaboration, and monitoring capabilities. This approach enables data scientists and ML engineers to focus on innovation.zenmlData bricksSafoine El khabich2024
- New Features: Enhanced Step Execution, AzureML Integration and More!ZenML's latest release 0.65.0 enhances MLOps workflows with single-step pipeline execution, AzureML SDK v2 integration, and dynamic model versioning. The update also introduces a new quickstart experience, improved logging, and better artifact handling. These features aim to streamline ML development, improve cloud integration, and boost efficiency for data science teams across local and cloud environments.AzureAzureMLrelease+4ZenML Team2024
- How to Finetune Phi 3.5 with ZenMLMaster cloud-based LLM finetuning: Set up infrastructure, run pipelines, and manage experiments with ZenML's Model Control Plane for Microsoft's latest Phi model.applied-zenmlfoundationmodelsmodel-control-plane+4Alex Strick van Linschoten2024
- Newsletter Edition #6 - Fine-tuning LLama 3.1 using your MLOps stackZenML's new direction: Simplifying infrastructure connections for enhanced MLOps.mlopsbest-practicesHamza Tahir2024
- How to Finetune Llama 3.1 with ZenMLMaster cloud-based LLM finetuning: Set up infrastructure, run pipelines, and manage experiments with ZenML's Model Control Plane for Meta's latest Llama model.applied-zenmlfoundationmodelsmodel-control-plane+4Alex Strick van Linschoten2024
- The Ultimate Guide to LLM Batch Inference with OpenAI and ZenMLOpenAI's Batch API allows you to submit queries for 50% of what you'd normally pay. Not all their models work with the service, but in many use cases this will save you lots of money on your LLM inference, just so long as you're not building a chatbot!machine-learningmlopsAlex Strick van Linschoten2024
- Newsletter Edition #4 - Learnings from Building with LLMsToday, we're back to LLM land (Not too far from Lalaland). Not only do we have a new LoRA + Accelerate-powered finetuning pipeline for you, we're also hosting a RAG themed webinar.zenmlmlopsHamza Tahir2024
- Bigger Isn't Always Better: The Case for RAG in the Age of Infinite ContextContext windows in large language models are getting super big, which makes you wonder if Retrieval-Augmented Generation (RAG) systems will still be useful. But even with unlimited context windows, RAG systems are likely here to stay because they're simple, efficient, flexible, and easy to understand.foundationmodelsThought LeadershipAlex Strick van Linschoten2024
- Newsletter Edition #2 - 2024: The most exciting year for MLOps yetnewslettermlopsHamza Tahir2024
- Newsletter Edition #1 - Welcome to the ZenML newsletter!newsletterHamza Tahir2024
- From RAGs to riches - The LLMOps pipelines you didn’t know you needed Taking large language models (LLMs) into production is no small task. It's a complex process, often misunderstood, and something we’d like to delve into today.Thought LeadershipHamza Tahir2024
- Using ZenML with LLMs to Analyze Your Databases: A Case Study with you-tldr.com and Supabase/GPT-4Explore how ZenML, an MLOps framework, can be used with large language models (LLMs) like GPT-4 to analyze and version data from databases like Supabase. In this case study, we examine the you-tldr.com website, showcasing ZenML pipelines asynchronously processing video data and generating summaries with GPT-4. Understand how to tackle large language model limitations by versioning data and comparing summaries to unlock your data's potential. Learn how this approach can be easily adapted to work with other databases and LLMs, providing flexibility and versatility for your specific needs.evergreentoolingzenml+1Hamza Tahir2023
- Productionalizing LangChain and LlamaIndex with a ZenML MLOps Pipeline to Help Community Slack SupportWe decided to explore how the emerging technologies around Large Language Models (LLMs) could seamlessly fit into ZenML's MLOps workflows and standards. We created and deployed a Slack bot to provide community support.evergreenfoundationmodelsnlp+3Alex Strick van Linschoten2023
Showing 1–12 of 44
AI orchestration,
on the infra you choose
Get new posts by email
New posts and product updates, sent when we publish. No spam.
By subscribing, you agree to our privacy policy.