What's New in v0.12.0: Serverless Inferencing on Kubernetes with KServe

Last updated: October 17, 2022.

The 0.12.0 release contains our KServe integration. KServe opens the door to highly scalable, simple, pluggable production ML serving.

We’ve been working on minor bug fixes and improving our docs page to improve your experience with ZenML.

We also added functionality to specify hardware resources on a step level to control the amount of memory, CPUs, and GPUs that each ZenML step has access to.

We’ve also added functionality to determine if secrets are shared with other ZenML Secrets Managers using the same backend.

As always, we’ve also included various bug fixes and lots of improvements to the documentation and our examples.

For a detailed look at what’s changed, give our full release notes a glance.

🌪 KServe Integration

The spotlight in this release is the ZenML Kserve integration. This integration lets you define pipeline steps to scale up your model by serving your ML models on Kubernetes using any of your favorite ML frameworks like TensorFlow, XGBoost, Scikit-Learn, PyTorch from within ZenML!

📌 Specify Resources Per Step

Some steps of your machine learning pipeline might be more resource-intensive and require special hardware to execute. In this release, you can now specify the resources (such as the amount of memory, CPU, and GPU) to allocate on a step level in your pipeline.

To allocate resources to a specific step in your pipeline, simply specify resource_configuration in the step decorator:

from zenml.steps import step, ResourceConfiguration
@step(resource_configuration=ResourceConfiguration(cpu_count=8, gpu_count=2))
def training_step(...) -> ...:
    # train a model

This currently works on KubeflowOrchestrator and VertexOrchestrator, but will be extended in the upcoming weeks to support the KubernetesOrchestrator.

🤫 Scoped Secrets

The majority of Secrets Managers use a single global scope in the back-end they are connected to. This means all secrets that are configured and visible through a Secrets Manager instance are also visible in all other instances. Furthermore, secrets configured directly in the back-end through means other than ZenML are also visible and can be managed in all ZenML Secrets Managers connected to that same back-end.

This has a number of disadvantages:

• There is only one global shared namespace for secret names. Two secrets with the same name and different values cannot be configured at the same time for different purposes (e.g. different ZenML projects or components).
• There is no clear separation between secrets managed by ZenML and those that are used for other purposes

In this release, we’ve added support for scoped secrets in our AWS, GCP, and Vault Secrets Managers. These updated Secrets Managers allow you to configure a scope that determines if secrets are shared with other ZenML Secrets Managers using the same backend.

➕ Other Updates, Additions, and Fixes

The latest release includes several smaller features and updates to existing functionality:

• Fix Links on the examples by @safoinme in https://github.com/zenml-io/zenml/pull/782
• Fix broken links in source code by @schustmi in https://github.com/zenml-io/zenml/pull/784
• Invalidating artifact/metadata store if there is a change in one of them by @bcdurak in https://github.com/zenml-io/zenml/pull/719
• Fixed broken link in README by @htahir1 in https://github.com/zenml-io/zenml/pull/785
• Embed Cheat Sheet in a separate docs page by @fa9r in https://github.com/zenml-io/zenml/pull/790
• Add data validation documentation by @stefannica in https://github.com/zenml-io/zenml/pull/789
• Add local path for mlflow experiment tracker by @schustmi in https://github.com/zenml-io/zenml/pull/786
• Improve Docker build logs. by @fa9r in https://github.com/zenml-io/zenml/pull/793
• Allow standard library types in steps by @stefannica in https://github.com/zenml-io/zenml/pull/799
• Added small description by @AlexejPenner in https://github.com/zenml-io/zenml/pull/801
• Replace the restriction to use Repository inside step with a warning by @stefannica in https://github.com/zenml-io/zenml/pull/792
• Adjust quickstart to data validators by @fa9r in https://github.com/zenml-io/zenml/pull/797
• Add utility function to deprecate pydantic attributes by @schustmi in https://github.com/zenml-io/zenml/pull/778
• Fix the mismatch KFP version between Kubeflow and GCP integration by @safoinme in https://github.com/zenml-io/zenml/pull/796
• Made mlflow more verbose by @htahir1 in https://github.com/zenml-io/zenml/pull/802
• Fix links by @dnth in https://github.com/zenml-io/zenml/pull/798
• KServe model deployer integration by @stefannica in https://github.com/zenml-io/zenml/pull/655
• retrieve pipeline requirement within running step by @safoinme in https://github.com/zenml-io/zenml/pull/805
• Fix --decouple_stores error message by @strickvl in https://github.com/zenml-io/zenml/pull/814
• Support subscripted generic step output types by @fa9r in https://github.com/zenml-io/zenml/pull/806
• Allow empty kubeconfig when using local kubeflow orchestrator by @schustmi in https://github.com/zenml-io/zenml/pull/809
• fix the secret register command in kserve docs page by @safoinme in https://github.com/zenml-io/zenml/pull/815
• Annotation example (+ stack component update) by @strickvl in https://github.com/zenml-io/zenml/pull/813
• Per-step resource configuration by @schustmi in https://github.com/zenml-io/zenml/pull/794
• Scoped secrets by @stefannica in https://github.com/zenml-io/zenml/pull/803
• Adjust examples and docs to new pipeline and step fetching syntax by @fa9r in https://github.com/zenml-io/zenml/pull/795

👩‍💻 Contribute to ZenML!

Join our Slack to let us know if you have an idea for a feature or something you’d like to contribute to the framework.

We have a new home for our roadmap where you can vote on your favorite upcoming feature or propose new ideas for what the core team should work on. You can vote without needing to log in, so please do let us know what you want us to build!

‍