A compiler-assisted approach to SPMD execution

Abstract

The two prevailing styles of scientific parallel programming are discussed. In the SPMD (single program, multiple data) style, all processors execute the same program, with sequential code executed redundantly and parallel code executed cooperatively. In the fork-join style, a sequential thread of control spawns multiple threads to execute a portion of the code concurrently. The authors describe an automatic method for approaching the efficiency of SPMD-style execution for programs written in the more-structured fork-join style. Analysis at compile-time and proper support at run-time yield execution efficiency that approaches the SPMD model. Moreover, a greater degree of portability is achieved by regulating the burden of deciding what should be in an SPMD parallel region to the compiler, which is probably more familiar with architectural detail than most programmers.