Gang scheduling for highly efficient distributed multiprocessor systems

Abstract

We have implemented a job scheduling system for workstation clusters and massively parallel systems with highly efficient message passing interconnects that supports space and time sharing through multiuser gang scheduling of parallel jobs. The system is available on the IBM-SP-2 cluster. It is highly modular, scalable and can easily be adapted to a variety of other MPP systems. The system supports various scheduling policies. We architect the system so that the time-sharing of processors avoids any significant serialization and extra resource consumption, but preserves the reliability and the efficiency of the high performance communication subsystem that characterizes a dedicated non time shared systems.