Software Engineering

What's New in v0.12.0: Serverless Inferencing on Kubernetes with KServe

Dickson Neoh Tze How
Aug 2, 2022
4 Mins Read

Last updated: October 17, 2022.

The 0.12.0 release contains our KServe integration. KServe opens the door to highly scalable, simple, pluggable production ML serving.

We’ve been working on minor bug fixes and improving our docs page to improve your experience with ZenML.

We also added functionality to specify hardware resources on a step level to control the amount of memory, CPUs, and GPUs that each ZenML step has access to.

We’ve also added functionality to determine if secrets are shared with other ZenML Secrets Managers using the same backend.

As always, we’ve also included various bug fixes and lots of improvements to the documentation and our examples.

For a detailed look at what’s changed, give our full release notes a glance.

🌪 KServe Integration

The spotlight in this release is the ZenML Kserve integration. This integration lets you define pipeline steps to scale up your model by serving your ML models on Kubernetes using any of your favorite ML frameworks like TensorFlow, XGBoost, Scikit-Learn, PyTorch from within ZenML!


📌 Specify Resources Per Step

Some steps of your machine learning pipeline might be more resource-intensive and require special hardware to execute. In this release, you can now specify the resources (such as the amount of memory, CPU, and GPU) to allocate on a step level in your pipeline.

To allocate resources to a specific step in your pipeline, simply specify resource_configuration in the step decorator:

from zenml.steps import step, ResourceConfiguration
@step(resource_configuration=ResourceConfiguration(cpu_count=8, gpu_count=2))
def training_step(...) -> ...:
    # train a model

This currently works on KubeflowOrchestrator and VertexOrchestrator, but will be extended in the upcoming weeks to support the KubernetesOrchestrator.

🤫 Scoped Secrets

The majority of Secrets Managers use a single global scope in the back-end they are connected to. This means all secrets that are configured and visible through a Secrets Manager instance are also visible in all other instances. Furthermore, secrets configured directly in the back-end through means other than ZenML are also visible and can be managed in all ZenML Secrets Managers connected to that same back-end.

This has a number of disadvantages:

  • There is only one global shared namespace for secret names. Two secrets with the same name and different values cannot be configured at the same time for different purposes (e.g. different ZenML projects or components).
  • There is no clear separation between secrets managed by ZenML and those that are used for other purposes

In this release, we’ve added support for scoped secrets in our AWS, GCP, and Vault Secrets Managers. These updated Secrets Managers allow you to configure a scope that determines if secrets are shared with other ZenML Secrets Managers using the same backend.

➕ Other Updates, Additions, and Fixes

The latest release includes several smaller features and updates to existing functionality:

  • Fix Links on the examples by @safoinme in
  • Fix broken links in source code by @schustmi in
  • Invalidating artifact/metadata store if there is a change in one of them by @bcdurak in
  • Fixed broken link in README by @htahir1 in
  • Embed Cheat Sheet in a separate docs page by @fa9r in
  • Add data validation documentation by @stefannica in
  • Add local path for mlflow experiment tracker by @schustmi in
  • Improve Docker build logs. by @fa9r in
  • Allow standard library types in steps by @stefannica in
  • Added small description by @AlexejPenner in
  • Replace the restriction to use Repository inside step with a warning by @stefannica in
  • Adjust quickstart to data validators by @fa9r in
  • Add utility function to deprecate pydantic attributes by @schustmi in
  • Fix the mismatch KFP version between Kubeflow and GCP integration by @safoinme in
  • Made mlflow more verbose by @htahir1 in
  • Fix links by @dnth in
  • KServe model deployer integration by @stefannica in
  • retrieve pipeline requirement within running step by @safoinme in
  • Fix --decouple_stores error message by @strickvl in
  • Support subscripted generic step output types by @fa9r in
  • Allow empty kubeconfig when using local kubeflow orchestrator by @schustmi in
  • fix the secret register command in kserve docs page by @safoinme in
  • Annotation example (+ stack component update) by @strickvl in
  • Per-step resource configuration by @schustmi in
  • Scoped secrets by @stefannica in
  • Adjust examples and docs to new pipeline and step fetching syntax by @fa9r in

👩‍💻 Contribute to ZenML!

Join our Slack to let us know if you have an idea for a feature or something you’d like to contribute to the framework.

We have a new home for our roadmap where you can vote on your favorite upcoming feature or propose new ideas for what the core team should work on. You can vote without needing to log in, so please do let us know what you want us to build!

Start your new ML Project today with ZenML Cloud

Join 1,000s of members already deploying models with ZenML.