IBM at PyTorch 2026

Speakers: Brian Stevens (Red Hat), Carlos Costa (IBM), & Abdullah Gharaibeh (Google Cloud)

https://events.linuxfoundation.org/pytorch-conference-north-america/program/schedule/?id=1302854

This talk presents how IBM Spyre streamlines its AI acceleration stack by leveraging PyTorch's open-source ecosystem. We showcase how PyTorch's Inductor compiler, combined with open-source extensions, provides a unified compilation framework that enables Spyre to be registered as an out-of-tree accelerator fully aligned with PyTorch's standardized interfaces. By integrating seamlessly with vLLM and adopting broader open-source model optimization techniques, we demonstrate a cohesive opensource approach that maximizes performance across diverse AI workloads on x86, IBM Z and Power platforms.

In this expo theater presentation, we'll demonstrate the new stack through two compelling use cases. First, we'll showcase an Insurance Claim Processing system leveraging Granite and Gemma models on IBM Z with Telum II processor, performing real-time multi-model AI inference for claim analysis and damage assessment including visual damage detection. Second, we'll present Document Intelligence with AI Optimizer, featuring multi-language support through IBM Granite Embedding Models, RAG-based document processing, and intelligent summarization capabilities across enterprise documents. Both demos highlight the practical impact of seamless PyTorch ecosystem integration for accelerating production AI applications.

https://events.linuxfoundation.org/pytorch-conference-north-america/program/schedule/?id=1351816

Torch-Spyre is an open source project that provides a PyTorch PrivateUse1 device with OpenReg, including an Inductor backend, for the IBM Spyre Accelerator. IBM Spyre is a high-performance energy-efficient AI accelerator featuring 32 AI-optimized compute cores each with on-chip interconnect and compiler-managed scratchpad memory.

The IBM Spyre Accelerator can now run thousands of models from Hugging Face and vLLM, thanks to the deep integration with PyTorch enabled via the PrivateUse1 interface and Inductor. The talk will cover the state of the Torch-Spyre project, describe key functional enablement and performance improvements made in 2026 and contributions made back to PyTorch to enable such a tight integration.

Providing the expected PyTorch programming model and user experience on a dataflow accelerator while also achieving our performance goals was challenging. We will discuss key items such as enabling device-specific tensor layouts and scratchpad optimized tiling, with an emphasis on how our solutions integrate with upstream PyTorch. We hope this will be of interest to the community and enable other out-of-tree accelerators to more easily achieve deep PyTorch integration.

Enabling new hardware, supporting thousands of models, and scaling inference takes progress across the open-source AI stack. This keynote shares how IBM and Red Hat are contributing across the PyTorch ecosystem to make that happen. We’ll highlight work bringing IBM Spyre to the PyTorch compiler and vLLM, how AI tools are helping scale model enablement to thousands of models, and a look ahead at next-generation Spyre hardware. We’ll also cover Red Hat’s contributions to PyTorch and CI/CD infrastructure for out-of-tree accelerators, helping hardware backends keep pace with the community. Finally, we will touch on our journey with the llm-d community to enable efficient distributed inference at scale.

Recently, the PyTorch team released Helion, a new domain-specific PyTorch-based language for developing high-performing, portable kernels. With extensive autotuning and broader backend support, Helion promises to advance performance portability beyond Triton.

To test this, we took on the challenge of writing one of AI's most performance-critical kernels in Helion: Paged Attention, the core of vLLM.

Building on our expertise creating the state-of-the-art portable Triton attention backend in vLLM, we developed the experimental Helion attention backend for vLLM.

In this talk, we will present the differences between paged attention in Triton and Helion, what changes to the algorithm of the kernel were necessary to achieve matching performance, and how Helion enables optimizations that were not possible with Triton. Then, we will share lessons we learned along the way and extensive benchmark results that show that Helion kernels can, in many scenarios, even outperform our optimized Triton kernels. Our early results indicate the experimental Helion attention backend can reduce the latency by up to 50% and improve the end-to-end throughput vs. Triton by up to 10%.