A reconfigurable optical/electrical interconnect architecture for large-scale clusters and datacenters

Abstract

Hybrid optical/electrical interconnects, using commercially available optical circuit switches at the core part of the network, have been recently proposed as an attractive alternative to fully-connected electronically-switched networks in terms of port density, bandwidth/port, cabling and energy efficiency. Although the shift from a traditionally packet-switched core to switching between server aggregations (or servers) at circuit granularity requires system redesign, the approach has been shown to fit well to the traffic requirements of certain classes of high-performance computing applications, as well as to the traffic patterns exhibited by typical data center workloads. Recent proposals for such system designs have looked at small/medium scale hybrid interconnects. In this paper, we present a hybrid optical/electrical interconnect architecture intended for large-scale deployments of high-performance computing systems and server co-locations. To reduce complexity, our architecture employs a regular shuffle network topology that allows for simple management and cabling. Thanks to using a single-stage core interconnect and multiple optical planes, our design can be both incrementally scaled up (in capacity) and scaled out (in the number of racks) without requiring major re-cabling and network re-configuration. Also, we are the first to our knowledge to explore the benefit of using multi-hopping in the optical domain as a means to avoid constant reconfiguration of optical circuit switches. We have prototyped our architecture at packet-level detail in a simulation framework to evaluate this concept. Our results demonstrate that our hybrid interconnect, by adapting to the changing nature of application traffic, can significantly exceed the throughput of a static interconnect of equal degree, while at times attaining a throughput comparable to that of a costly fully-connected network. We also show a further benefit brought by multi-hopping, that it reduces the performance drops by reducing the frequency of reconfiguration. © 2012 ACM.