Integrating the IBM Spyre accelerator

Torch-Spyre is an open source project that provides a PyTorch PrivateUse1 device with OpenReg, including an Inductor backend, for the IBM Spyre Accelerator. IBM Spyre is a high-performance energy-efficient AI accelerator featuring 32 AI-optimized compute cores each with on-chip interconnect and compiler-managed scratchpad memory.

The IBM Spyre Accelerator can now run thousands of models from Hugging Face and vLLM, thanks to the deep integration with PyTorch enabled via the PrivateUse1 interface and Inductor. The talk will cover the state of the Torch-Spyre project, describe key functional enablement and performance improvements made in 2026 and contributions made back to PyTorch to enable such a tight integration.

Providing the expected PyTorch programming model and user experience on a dataflow accelerator while also achieving our performance goals was challenging. We will discuss key items such as enabling device-specific tensor layouts and scratchpad optimized tiling, with an emphasis on how our solutions integrate with upstream PyTorch. We hope this will be of interest to the community and enable other out-of-tree accelerators to more easily achieve deep PyTorch integration.