Last updated: February 1, 2023
Introduction
This project showcases a simple pipeline on how you can use a local orchestrator and train a Scikit-learn model remotely on VertexAI using a step operator. The artifacts are stored on a Google Cloud Storage bucket and the docker images are stored on Google Container Registry.
We will construct a pipeline for this project which loads the data, trains a model, and trains a model remotely in VertexAI.
Use case
This project structure including the stack and components can be used on occasions you need to construct ML pipelines for tabular or time series data ML problems that require remote infrastructure for training.
Stack and Components
This project uses the following Stack Components:
- Orchestrator - Local Orchestrator.
- Artifact Store - Google Cloud Storage.
- Container Registry - Google Container Registry.
- Step Operator - VertexAI.
Code
The codes to reproduce this project are open-source ZenML Project repository on GitHub. View the code here.
Runs
The pipeline was run with a local orchestrator and a remote step operator to train the model on VertexAI. The visualization is shown on the ZenML Dashboard.
The following DAG shows the pipeline on the ZenML Dashboard:
