BentoML

Simplify your model deployment process by integrating BentoML with ZenML. This powerful combination allows you to package models into production-ready Bentos and serve them locally or in the cloud with ease, streamlining the transition from development to production.

Features with ZenML

• Streamlined Model Packaging: Effortlessly package trained models into Bentos using ZenML's built-in BentoML steps
• Local Model Serving: Deploy and serve models locally for development and testing with the BentoML Model Deployer
• Container-based Model Serving: ZenML’s in-built steps convert your bento into Docker images and automatically push them to your Stack’s Container Registry, from where you can deploy them anywhere.
• Cloud Deployment Ready: Bentos are versioned and tracked and you can fetch them from ZenML for seamless deployment to various cloud platforms using bentoctl  or yatai .
• Standardized Deployment Workflow: Establish a consistent and reproducible model deployment process across your organization

Main Features

• Framework-agnostic model packaging and serving
• Supports local, cloud, and Kubernetes deployments
• Easy-to-use Python API for defining prediction services
• Automatic generation of OpenAPI specifications
• Built-in monitoring and logging capabilities

How to use ZenML with BentoML

zenml model-deployer register bentoml_deployer --flavor=bentoml
zenml stack update -d bentoml_deployer

You first need to define a BentoML Service in a service.py file and define the logic to serve your model there. It could look like the following:

@bentoml.service(
    name=SERVICE_NAME,
)
class MNISTService:
    def __init__(self):
        # load model
        self.model = bentoml.pytorch.load_model(MODEL_NAME)
        self.model.eval()

    @bentoml.api()
    async def predict_ndarray(
        self, 
        inp: Annotated[np.ndarray, DType("float32"), Shape((28, 28))]
    ) -> np.ndarray:
        inp = np.expand_dims(inp, (0, 1))
        output_tensor = await self.model(torch.tensor(inp))
        return to_numpy(output_tensor)

You can then define your pipeline as follows:

from zenml import pipeline, step
from zenml.integrations.bentoml.steps import bento_builder_step
from zenml.integrations.bentoml.steps import bentoml_model_deployer_step


@pipeline
def bento_builder_pipeline():
    model = model_training_step()
    bento = bento_builder_step(
        model=model,
        model_name="pytorch_mnist",  # Name of the model
        model_type="pytorch",  # Type of the model (pytorch, tensorflow, sklearn, xgboost..)
        service="service.py:CLASS_NAME",  # Path to the service file within zenml repo
    )
    deployed_model = bentoml_model_deployer_step(
        bento=bento,
        model_name="pytorch_mnist",  # Name of the model
        port=3001,  # Port to be used by the http server
        deployment_type="container" # the type of deployment, either local or container
    )

Additional Resources

• Official BentoML Documentation
• BentoML ZenML SDK Docs