Building AutoML Pipelines With Argo Workflows and Katib - Andrey Velichkevich + Johnu George

Описание: Building AutoML Pipelines With Argo Workflows and Katib - Andrey Velichkevich, Apple + Johnu George, Nutanix

The fairly recent field of Automated Machine Learning (AutoML) provides the richness of powerful algorithms for model selection and hyperparameter (HP) tuning – one of the most important steps of the MLOps lifecycle. Katib is a popular Kubernetes native open source project to perform AutoML. Katib can tune HPs for models written in any framework such as Tensorflow, PyTorch, MXNet, and Scikit learn. To find the best HPs, metrics are evaluated after a model training step. Usually, the model training is a complex process which includes data preprocessing, data validation, actual training, and many more. This whole lifecycle can be represented by a workflow dependency graph by specifying dependencies between model operations. Argo Workflows provides a great container-native workflow engine to orchestrate jobs on Kubernetes, which makes it an ideal candidate for Katib Experiments. This talk will demonstrate how Argo Workflows natively integrates in Katib infrastructure.