Statistical approach to diagnosing intermittent performance-problems using monotone relationships

Abstract

This paper describes a statistical approach to diagnosing intermittent performance problems when the relationships among measurement variables are expressed qualitatively as monotone relationships (e.g., paging delays increase with the number of logged-on users. We present a nonparametric test for monotonicity (NTM) that evaluates monotone relationships based on FA, the fraction of observation-pairs that agree with the monotone relationship. An interpretation of FA in terms of statistical significance levels is presented, and NTM is compared to least-squares regression. Based on NTM, an algorithm for diagnosing intermittent performance-problems is presented. NTM and our diagnosis algorithm are applied to measurements of four similarly configured IBM 9370 model 60s running IBM's operating-system Virtual Machine System Product (VM SP).