Improvements: Enhanced Artifacts Versioning, Scalability and Metadata Management

ZenML 0.70.0 has launched with major improvements but requires careful handling during upgrade due to significant database schema changes. Key highlights include enhanced artifact versioning with batch processing capabilities, improved scalability through reduced server requests, unified metadata management via the new log_metadata method, and flexible filtering with the new oneof operator. The release also features expanded documentation covering finetuning and LLM/ML engineering resources. Due to the database changes, users must back up their data and test the upgrade in a non-production environment before deploying to production systems.

ZenML 0.70.0: Major Updates & Important Migration Notes

We’re excited to announce the release of ZenML 0.70.0, which brings significant improvements to artifact versioning, scalability, and metadata management. However, due to substantial database schema changes, this upgrade requires careful attention and preparation.

⚠️ Important Upgrade Notice

This release includes major database schema changes and migrations. Before proceeding with the upgrade:

Create a complete backup of your production database
Test the upgrade process in a non-production environment first
Plan for potential downtime during the migration

🚀 Key Improvements

Enhanced Artifact Versioning

We’ve substantially improved how ZenML handles artifact versions:

New batch processing capabilities for artifact version requests, resulting in faster execution times
Support for multiple versions of the same artifact (e.g., model checkpoints)
Improved user experience in both the ZenML UI and API interactions

Scalability Boost

Performance and scalability have been major focus areas in this release:

Reduced server request frequency
Server-side artifact version incrementing
Optimized processing for improved speed at scale

Unified Metadata Management

We’ve simplified metadata handling with a new unified approach:

Introduced the consolidated log_metadata method
Support for logging run metadata across:
- Artifact versions
- Model versions
- Steps
- Runs

New Filtering Capabilities

The new oneof operator brings more flexible filtering options:

“# Filter by IDs (UUID type)PipelineRunFilter(id='oneof:["uuid1", "uuid2"]')# Filter by tags or string attributesPipelineRunFilter(tag='oneof:["cats", "dogs"]')

Documentation Enhancements

Our documentation has received a significant update:

Restructured for better navigation
New sections covering finetuning
Additional LLM/ML engineering resources
Improved getting started guides

Bug Fixes

This release addresses several issues:

Resolved in-process main module source loading problems
Various other stability improvements

📋 Upgrade Checklist

Before upgrading to version 0.70.0:

Create a full backup of your ZenML database
Test the upgrade in a staging environment
Review your current workflows for any potential compatibility issues
Plan for maintenance window if upgrading production systems
Ensure all team members are aware of the upcoming changes

Looking Forward

These improvements lay the groundwork for even better ML pipeline management and scalability. We’re committed to making ZenML more robust and user-friendly with each release.

Stay tuned for more updates, and don’t hesitate to reach out to our community channels if you need any assistance with the upgrade process.

Happy MLOps!

release release-notes zenml mlops

ZenML

Scaling ZenML: 200x Performance Improvement Through Database and FastAPI Optimizations in v0.83.0

A technical deep dive into the performance optimizations that improved ZenML's throughput by 200x

Hamza Tahir

15 mins

ZenML Updates

ZenML 0.80.0: Workspace Hierarchy for Pro, Performance Gains for All

ZenML 0.80.0 transforms tenant structures into workspace/project hierarchies with advanced RBAC for Pro users, while enhancing tagging, resource filtering, and dashboard design. Open-source improvements include Kubernetes security upgrades, SkyPilot integration, and significantly faster CLI operations. Both Pro and OSS users benefit from dramatic performance optimizations, GitLab improvements, and enhanced build tracking.