Scalable power management for AI-centric SoCs

Abstract

Modern day System-on-Chip (SoCs) are being designed for increasingly complex applications, particularly larger and more diverse AI models. This, in turn has, led to them becoming much larger, and more heterogeneous, with tens to hundreds of cores and accelerators which has necessitated a rethink of power and resource management techniques on these SoCs. Existing power-management strategies with centralized on-chip controllers are becoming more and more difficult to translate to current or next-generation SoCs due to tight constraints on response time and scalability. With this in mind, we present EPOCHS-1, a large heterogeneous SoC fabricated in 12nm technology, comprising of 34 PE tiles with 14 different accelerator types, equipped with a novel, decentralized hardware power management strategy, termed as BlitzCoin. We demonstrate through a combination of RTL simulations and actual chip measurements, an 8-12X improvement in response time coupled with up to 34% throughput improvement, compared with existing state-of-the-art centralized power management schemes. Finally we also demonstrate a novel hybrid voltage regulator design with the required response time and efficiency to achieve these benefits.