mlflow-spark-summit-2019

MLflow code for Spark Summit 2019.

Session: Managing the Complete Machine Learning Lifecycle with MLflow.

Setup

pip install mlflow==0.9.1
pip install matplotlib
pip install pyarrow

MLflow Server

virtualenv mlflow_server
source mlflow_server/bin/activate
mlflow server --host 0.0.0.0 --port 5000 --backend-store-uri $PWD/mlruns --default-artifact-root $PWD/mlruns

Examples

Before running an experiment:

export MLFLOW_TRACKING_URI=http://localhost:5000

• hello_world - Hello World
• sklearn - Scikit learn model
• pyspark - PySpark model
• scala_spark - Scala Spark ML model using the Java client
• search - Shows new MLflow 0.9.1 Search feature
• dump - Shows usage of some mlflow.tracking package methods
• best_run - Finds the best model run