Dslash: Managing Data in Overloaded Batch Streaming Systems

Abstract

Peak loads are a challenge for streaming systems. Here we present Dslash, a latency-driven controller that keeps and processes as much data as the system resources allow in order to meet a target latency.