Building and Optimizing RAG Pipelines: Data Preprocessing, Embeddings, and Evaluation with ZenML
We dive deep into the world of Retrieval-Augmented Generation (RAG) pipelines and how ZenML can streamline your RAG workflows.
Jun 14, 20242 mins
We dive deep into the world of Retrieval-Augmented Generation (RAG) pipelines and how ZenML can streamline your RAG workflows.
Today, we're back to LLM land (Not too far from Lalaland). Not only do we have a new LoRA + Accelerate-powered finetuning pipeline for you, we're also hosting a RAG themed webinar.
Context windows in large language models are getting super big, which makes you wonder if Retrieval-Augmented Generation (RAG) systems will still be useful. But even with unlimited context windows, RAG systems are likely here to stay because they're simple, efficient, flexible, and easy to understand.
We've open-sourced our new dashboard to unify the experience for OSS and cloud users, although some features are initially CLI-only. This launch enhances onboarding and simplifies maintenance. Cloud users will see no change, while OSS users can enjoy a new interface and DAG visualizer. We encourage community contributions to help us expand and refine this dashboard further, looking forward to integrating more features soon.
Community member Marwan Zaarab explains how and why he built a VS Code Extension for ZenML.
Taking large language models (LLMs) into production is no small task. It's a complex process, often misunderstood, and something we’d like to delve into today.
A critical security vulnerability has been identified in ZenML versions prior to 0.46.7. This vulnerability potentially allows unauthorized users to take ownership of ZenML accounts through the user activation feature.
Seamlessly automating the journey from training to production, ZenML's new NLP project template offers a comprehensive MLOps solution for teams deploying Huggingface models to AWS Sagemaker endpoints. With its focus on reproducibility, scalability, and best practices, the template simplifies the integration of NLP models into workflows, complete with lineage tracking and various deployment options.
