Leveraging Argo Workflows and Argo Events for Cloud Agnostic Enter... Charles Adetiloye & Alex Lerma

Описание: Leveraging Argo Workflows and Argo Events for Cloud Agnostic Enterprise Machine Learning Deployments and Model Management - Charles Adetiloye, MavenCode & Alex Lerma, MavenCode

The goal of every organization integrating Machine Learning into their product pipeline is to get it into production as fast as possible using standard software engineering best practices, however, in most cases, this is not often the reality. Even when the models are successfully deployed there is always a need for an efficient workflow to monitor for things like model drift, model overfitting, or constantly changing datasets. To solve this recurring problem in our MLOps practice and create a reusable pipeline and solution framework, we have explored and implemented best practices around Argo Workflows and Argo Events. In this talk, we will discuss how we have implemented a continuous deployment environment stack to containerize and deploy Argo workflows for data ingestion, transformation, and feature store curation. We will also talk about how we trigger model retraining and redeployment with the Argo workflow pipeline. This includes scheduled events, message queue updates, and external storage data changes. We will go through the challenges we encountered and lessons learned with recommended best practices for any MLOps team considering the Argo workflow approach.