Streamline your MLOps workflows on GCP with ZenML and Vertex AI Pipelines

Enhance your machine learning operations by leveraging the power of Vertex AI Pipelines orchestration through ZenML. This integration enables you to run production-ready, scalable ML pipelines on Google Cloud Platform, taking advantage of the fully managed serverless infrastructure and intuitive UI for tracking pipeline runs.

Features with ZenML

Seamlessly integrate ZenML pipelines with Vertex AI Pipelines for end-to-end ML workflows on GCP
Easily deploy and scale your pipelines using Vertex AI's managed serverless infrastructure
Track and monitor pipeline runs through the intuitive Vertex AI UI, accessible directly from ZenML
Leverage GPU acceleration for compute-intensive steps in your ZenML pipelines
Schedule recurring pipeline runs using Vertex AI's native scheduling capabilities

‍

Main Features

Fully managed serverless infrastructure for running ML pipelines at scale
Intuitive UI for visualizing and monitoring pipeline runs and logs
Native support for GPU-accelerated workloads
Flexible scheduling options for recurring pipeline runs
Seamless integration with other GCP services and tools in the Vertex AI platform

‍

How to use ZenML with

Google Cloud Vertex AI Pipelines


zenml integration install gcp
zenml stack set ...


from zenml.integrations.gcp.flavors.vertex_orchestrator_flavor import VertexOrchestratorSettings

# Choose an accelerator to run on
vertex_settings = VertexOrchestratorSettings(
                node_selector_constraint=(
                    "cloud.google.com/gke-accelerator",
                    "NVIDIA_TESLA_P4",
                )
            )

@pipeline(
    settings={
        "orchestrator.vertex": vertex_settings,
    }
)
def vertex_pipeline():
    ingest_data()
    train_model()
    evaluate_model()

# Run the pipeline
vertex_pipeline()

This code snippet demonstrates how to configure a ZenML pipeline to run on Vertex AI Pipelines. The VertexOrchestratorSettings allows you to specify settings and other settings for the Vertex AI job. The pipeline steps are defined as usual, and the pipeline is run by calling vertex_pipeline().

Additional Resources

Read the full Vertex AI Pipelines integration documentation

Using Vertex with a full google stack

Learn more about Vertex AI Pipelines in the GCP documentation