| Workflow Orchestration | Portable, code-defined pipelines that run on any orchestrator (Airflow, Kubeflow, local, etc.) via composable stacks | Built-in visual Flow orchestrator with Scenarios for scheduling, event triggers, and conditional automation |
| Integration Flexibility | Designed to integrate with any ML tool — swap orchestrators, trackers, artifact stores, and deployers without changing pipeline code | Rich built-in connectors (40+ data sources) and plugins, but integrations work within Dataiku's platform abstraction layer |
| Vendor Lock-In | Open-source and vendor-neutral — pipelines are pure Python code portable across any infrastructure | Proprietary platform where visual Flows, Recipes, and Scenarios are tied to Dataiku DSS — migrating away requires reimplementation |
| Setup Complexity | Pip-installable, start locally with minimal infrastructure — scale by connecting to cloud compute when ready | Enterprise setup requires Design, Automation, and API nodes with server provisioning. Cloud trial available but production is heavyweight |
| Learning Curve | Familiar Python pipeline definitions with simple decorators — fewer platform concepts to learn for ML engineers | Visual interface accessible to non-coders (analysts, business users). Extensive Academy training. But mastering the full platform takes time |
| Scalability | Scales via underlying orchestrator and infrastructure — leverage Kubernetes, cloud services, or distributed compute | Enterprise-grade scaling with in-database SQL push-down, Spark integration, Kubernetes execution, and multi-node architecture |
| Cost Model | Open-source core is free — pay only for infrastructure. Optional managed service with transparent usage-based pricing | Enterprise subscription pricing (sales-led, custom quotes). Free Edition available for up to 3 users with limited production features |
| Collaborative Development | Collaboration through code sharing, Git workflows, and the ZenML dashboard for pipeline visibility and model management | Strong multi-persona collaboration with project wikis, discussions, shared dashboards, and role-based access across data scientists and analysts |
| ML Framework Support | Framework-agnostic — use any Python ML library in pipeline steps with automatic artifact serialization | Built-in AutoML covers scikit-learn, XGBoost, and TensorFlow/Keras. Code recipes support any framework installable in code environments |
| Model Monitoring & Drift Detection | Integrates with monitoring tools like Evidently and Great Expectations as pipeline steps for customizable drift detection | Built-in Model Evaluation Store, Unified Monitoring dashboard, and drift analysis for data, prediction, and performance drift |
| Governance & Access Control | Pipeline-level lineage, artifact tracking, RBAC, and model control plane for audit trails and approval workflows | Enterprise-grade governance with Dataiku Govern module, audit logs, data catalog and lineage, LDAP/SSO, and regulatory compliance features |
| Experiment Tracking | Integrates with any experiment tracker (MLflow, W&B, etc.) as part of your composable stack | Built-in experiment tracking for AutoML with model comparison UI. Supports logging from scikit-learn, XGBoost, LightGBM, and TensorFlow |
| Reproducibility | Auto-versioned code, data, and artifacts for every pipeline run — portable reproducibility across any infrastructure | Managed code environments, project bundles for deployment, and Flow determinism. Requires discipline around data versioning |
| Auto Retraining Triggers | Supports scheduled pipelines and event-driven triggers that can initiate retraining based on drift detection or data changes | Native Scenarios with time-based schedules, event triggers, and conditional logic for automated retraining and deployment |
from zenml import pipeline, step, Model
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
import pandas as pd
@step
def ingest_data() -> pd.DataFrame:
return pd.read_csv("data/dataset.csv")
@step
def train_model(df: pd.DataFrame) -> RandomForestClassifier:
X, y = df.drop("target", axis=1), df["target"]
model = RandomForestClassifier(n_estimators=100)
model.fit(X, y)
return model
@step
def evaluate(model: RandomForestClassifier, df: pd.DataFrame) -> float:
X, y = df.drop("target", axis=1), df["target"]
return float(accuracy_score(y, model.predict(X)))
@step
def check_drift(df: pd.DataFrame) -> bool:
# Plug in Evidently, Great Expectations, etc.
return detect_drift(df)
@pipeline(model=Model(name="my_model"))
def ml_pipeline():
df = ingest_data()
model = train_model(df)
accuracy = evaluate(model, df)
drift = check_drift(df)
# Runs on any orchestrator (local, Airflow, Kubeflow),
# auto-versions all artifacts, and stays fully portable
# across clouds — no platform lock-in
ml_pipeline()# Dataiku DSS platform workflow
# Runs inside Dataiku's managed environment
import dataiku
import pandas as pd
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
# Read input dataset from Dataiku's managed storage
dataset = dataiku.Dataset("customers_prepared")
df = dataset.get_dataframe()
X = df.drop("target", axis=1)
y = df["target"]
# Train model inside Dataiku's code recipe
model = RandomForestClassifier(n_estimators=100)
model.fit(X, y)
acc = accuracy_score(y, model.predict(X))
print(f"Accuracy: {acc}")
# Write predictions to output Dataiku dataset
preds = pd.DataFrame({"prediction": model.predict(X)})
output = dataiku.Dataset("predictions")
output.write_with_schema(preds)
# Multi-step orchestration uses visual Flows + Scenarios
# (configured through Dataiku's platform UI).
# AutoML, monitoring, and retraining are all managed
# within the proprietary DSS environment.
# Requires Dataiku server and enterprise license.
