Rules of thumb for selecting metrics for detecting performance problems

Abstract

This paper addresses the selection of metrics to facilitate the early detection of performance problems. Our approach combines results from queueing theory and statistical hypothesis testing to develop rules-of-thumb for when one metric is preferred to another. Examples of these rules include: (1) measures of queue length are more sensitive to performance problems than are measures of utilization; (2) queue length measures provide more sensitive detection than measures of response times if the performance problem is dominated by an increase in expected arrival rates; and (3) response times are preferred to queue lengths if the performance problem is dominated by an increase in expected service times. These rules are assessed for performance problems in the CPU and paging sub-systems of a production computer system. In all cases, the data are consistent with the rules.