Adaptive SMT control for more responsive web applications

Abstract

We developed a new task scheduling technique that improves the response times of Web applications. Most of the high-performance processors used in today's servers support thread-level parallelism with multiple hardware threads within each core using Simultaneous Multi-Threading (SMT). SMT capabilities can improve the peak throughput of a server by increasing the utilization of the computing resources in the processor. However, SMT may degrade the response time of Web applications if the server does not fully utilize its multicore CPUs, while SMT improves the response times regardless of the CPU utilization if the server is equipped with only one core. To precisely model this behavior, we created a new hierarchical queuing model to accurately predict the response times on multicore SMT processors, taking the CPU utilization and the number of CPU cores into account. Based on this model, we devised Adaptive SMT control, a technique to control the number of active SMT threads (SMT level) to minimize the response time. We measure the current CPU utilization and the number of cores, predict the response time for each SMT level and dynamically set the SMT level that yields the best response times for the Web applications. We show our technique successfully improves the response time of Web applications written in PHP, Ruby, and Java by up to 12.9% on Xeon, which employs 2-way SMT, and 12.4% on POWER7, which employs 4-way SMT, when the CPUs are not fully utilized. It is known that the CPU utilization levels are typically low in many commercial servers. Hence our new technique can improve the response times of many Web applications, thus improving the users' experience.