Towards an understanding of decision complexity in IT configuration

Abstract

In previous work we laid out an approach to quantifying configuration complexity [3]. In that earlier work, we explicitly focused on complexity as experienced by expert systems managers, and thus looked at straight-line configuration procedures, ignoring the complexity faced by non-experts as they have to decide what configuration steps to follow. Decision complexity is the complexity faced by a non-expert system administrator - -the person providing IT support in a small-business environment, who is confronted by decisions during the configuration process, and is a measure of how easy or hard it is to identify the appropriate sequence of configuration actions to perform in order to achieve a specified configuration goal. To identify spots of high decisionmaking complexity, we need a model of decision complexity for configuring and operating computing systems. This paper extends previous work on models and metrics for IT configuration complexity by adding the concept of decision complexity. As the first step towards a complete model of decision complexity, we describe an extensive user study of decision making in a carefully-mapped analogous domain (route planning), and illustrate how the results of that study suggest an initial model of decision complexity applicable to IT configuration. The model identifies the key factors affecting decision complexity and highlights several interesting results, including the fact that decision complexity has significantly different impacts on user-perceived difficulty than on objective measures like time and error rate. We also describe some of the implications of our decision complexity model for system designers seeking to automate the decision-making and reduce the configuration complexity of their systems. Copyright 2007 ACM.