End-to-End AI (E2EAI) with Julia, K0s, and Argo Workflow

Abstract

This talk will demonstrate an End-to-End AI (E2EAI) implementation by leveraging on Julia’s AutoMLPipeline package for developing AI solution together with K0s and Argo Workflow frameworks to automate the provisioning of the K8s cluster and the deployment and monitoring of AI solutions. Current implementations in cloud computing treat MLOps and IaC (infrastructure as a code) as separate activities. Without a unified view of both the needed infrastructure and the data processing and modelling tasks involved, current approaches put the burden of integrating the automation of both AI solutions and infrastructure to majority of stakeholders that typically lack the specialization of both technologies at the same time. E2EAI (End-to-End AI) aims to create a unified framework by tightly integrating MLOps and IaC workflows using high-level yaml description and APIs for zero to minimal coding leveraging on Julia’s performance and efficiency together with K0s high-level abstractions of k8s infrastructure and Argo Workflow’s ML pipeline stages of deploying micro-services.