KubeFlux: An HPC scheduler plugin for Kubernetes

Abstract

Adoption of cloud technologies by high performance computing (HPC) is accelerating, and HPC users want their applications to perform well everywhere. While container orchestration frameworks provide advantages like resiliency, elasticity, and declarative management, they are not designed to enable application performance to the same degree as HPC workload managers and schedulers. In response to increased interest in scheduling flexibility, the Kubernetes community developed the Scheduling Framework to facilitate integration of new policies and schedulers. We present KubeFlux, a Scheduling Framework plugin based on the Fluxion open-source HPC scheduler developed at the Lawrence Livermore National Laboratory. We discuss uses for KubeFlux and compare the performance of an application scheduled by the Kubernetes default scheduler and KubeFlux. KubeFlux is an example of the rich capability that can be added to Kubernetes and paves the way to democratization of the cloud for HPC workloads.