Personalized Communication Avoiding Node Contention on Distributed Memory Systems

Abstract

In this paper, we present several algorithms for per forming all-to-many personalized communication on distributed memory parallel machines. Each proces sor sends a different message (of potentially different size) to a subset of all the processors involved in the collective communication. The algorithms are based on decomposing the communication matrix into a set of partial permutations. We study the effectiveness of our algorithms both from the view of static scheduling as well as runtime scheduling.