Track Model Pipelines

When making business decisions based on machine learning models, it is common to use a number of models chained together as a pipeline to arrive to a final decision. MPM allows you monitor not just models but also model pipelines.

MPM Pipelines tracks the performance of both individual models and the pipeline as a whole.

Defining Model Pipelines¶

The MPM UI includes an interactive pipeline builder that can be used to define pipelines. It is accessible though the Create DAG button in the MPM model list page.

To define the pipeline, simple drag and drop each model into the right hand side of the editor and link them together to define the DAG. To finish creating the pipeline, you will also need to define the models that are inputs to the pipeline and the one that is the output of the pipeline as well as pipeline name.

Viewing Model Pipeline metrics¶

When monitoring model pipelines, it is important to consider both pipeline level metrics and model level metrics. By tracking pipeline level metrics you will be able to detect any issues with the final prediction that is being made. When an issue does arise with a pipeline it is often because one the models within this pipeline is experiencing data drift, by having access to model level metrics you are able easily identify the source of the drift.

Once a pipeline has been defined, you will be able to view it in the model list page. By clicking on the name of the pipeline, you will be able to see the pipeline level metrics. The input and output features at a pipeline features are defined based on the models marked as input models and output model during pipeline definition:

• Pipeline input features: Input features of the models marked as input model
• Pipeline output features: Prediction value and probability for the model marked as output model