Autonomic query parallelization using non-dedicated computers: An evaluation of adaptivity options

Abstract

Writing parallel programs that can take advantage of non-dedicated processors is much more difficult than writing such programs for networks of dedicated processors. In a non-dedicated environment such programs must use autonomic techniques to respond to the unpredictable load fluctuations that characterize the computational model. In the area of adaptive query processing (AQP), several techniques have been proposed for dynamically redistributing processor assignments throughout a computation to take account of varying resource capabilities, but we know of no previous study that compares their performance. This paper presents an simulation based evaluation of these autonomic parallelization techniques in a uniform environment and compares on how well they improve the scaleability of the computation. The existing strategies are compared with a new approach inspired by work on distributed hash tables. The evaluation identifies situations in which each strategy may be used effectively and it should be avoided. © 2006 IEEE.