Fast Modeling of Analytics Workloads for Big Data Services

Abstract

Building models to predict analytics workloads' execution is a foundational capability that enables key scenarios for big data services, like SLA-driven service provisioning and elastic auto scaling. Given the various infrastructure and workload characteristics, it's more preferable to build the models in a 'black-box' fashion, for example, by leveraging machine learning techniques. However, this approach has assumptions on the volume and quality of workloads' existing records to learn from, which require sophisticate benchmark or long time monitoring. In this paper, we present a method to accelerate the modeling process of an analytics workload for its quick time-to-value in the context of big data services. Specifically, clustering and transfer learning techniques are leveraged for this acceleration by shifting the data collection from the online service phase to the offline preparation phase. This paper focuses on the conceived service model and fast modeling techniques. Their feasibility is demonstrated by experiments.