CLOUD 2022
Conference paper

Detecting Layered Bottlenecks in Microservices


We propose a method to detect both software and hardware bottlenecks in a web service consisting microservices. A bottleneck is a resource which limits the maximum performance of the entire service. Bottlenecks often include both software resources such as threads, locks, and channels, and hardware resources such as processors, memories, and disks. Bottlenecks form a layered structure since a single request can utilize multiple software resources and a hardware resource simultaneously. The microservice architecture makes detection of layered bottlenecks challenging due to the lack of a uniform analyzing perspective across languages, libraries, frameworks, and middleware. We detect layered bottlenecks in microservices by profiling numbers and status of working threads in each microservice and dependency among microservices via network connections. Our approach is applicable to various programming languages since it relies only on standard debugging tools. Nevertheless, our approach detects not only which microservice is a bottleneck, but also allows us to understand why it becomes a bottleneck. This is enabled by a novel visualization method to show layered bottlenecks in microservices at a glance. We demonstrate that our approach successfully detects and visualizes layered bottlenecks in state-of-the-art microservice benchmarks, DeathStarBench and the AcmeAir microservices. This allows us to optimize the microservices themselves to achieve a higher throughput per resource utilization rate compared to simply scaling the number of replicas of microservices.