IBM at PyTorch 2026

Speakers: Brian Stevens (Red Hat), Carlos Costa (IBM), & Abdullah Gharaibeh (Google Cloud)

https://events.linuxfoundation.org/pytorch-conference-north-america/program/schedule/?id=1302854

This talk presents how IBM Spyre streamlines its AI acceleration stack by leveraging PyTorch's open-source ecosystem. We showcase how PyTorch's Inductor compiler, combined with open-source extensions, provides a unified compilation framework that enables Spyre to be registered as an out-of-tree accelerator fully aligned with PyTorch's standardized interfaces. By integrating seamlessly with vLLM and adopting broader open-source model optimization techniques, we demonstrate a cohesive opensource approach that maximizes performance across diverse AI workloads on x86, IBM Z and Power platforms.

In this expo theater presentation, we'll demonstrate the new stack through two compelling use cases. First, we'll showcase an Insurance Claim Processing system leveraging Granite and Gemma models on IBM Z with Telum II processor, performing real-time multi-model AI inference for claim analysis and damage assessment including visual damage detection. Second, we'll present Document Intelligence with AI Optimizer, featuring multi-language support through IBM Granite Embedding Models, RAG-based document processing, and intelligent summarization capabilities across enterprise documents. Both demos highlight the practical impact of seamless PyTorch ecosystem integration for accelerating production AI applications.

https://events.linuxfoundation.org/pytorch-conference-north-america/program/schedule/?id=1351816

As models grow, startup time has become a bottleneck for PyTorch inference systems like vLLM. A large model can take minutes to load, which slows down experimentation, deployment, autoscaling, and failure recovery.

This talk presents fastsafetensors, an open-source library that accelerates safetensors checkpoint loading. fastsafetensors treats loading as a data-movement problem: it removes per-tensor copies, coalesces fragmented I/O, and skips host staging. The result is 4.8x to 7.5x faster model loading and up to 28 GB/s of NVMe read throughput. It already plugs into existing LLM serving systems, for example:

vllm serve Qwen/Qwen3-0.6B --load-format fastsafetensors

We will highlight community contributions that have transformed fastsafetensors from its original NVIDIA GPU-specific loader into a portable framework: parallel loading, 3FS integration, ROCm support, a universal wheel with runtime CUDA/ROCm detection, Windows DirectStorage exploration, and unified-memory support.

Attendees will learn how fastsafetensors works and how to make model loading fast, portable, and practical.

Torch-Spyre is an open source project that provides a PyTorch PrivateUse1 device with OpenReg, including an Inductor backend, for the IBM Spyre Accelerator. IBM Spyre is a high-performance energy-efficient AI accelerator featuring 32 AI-optimized compute cores each with on-chip interconnect and compiler-managed scratchpad memory.

The IBM Spyre Accelerator can now run thousands of models from Hugging Face and vLLM, thanks to the deep integration with PyTorch enabled via the PrivateUse1 interface and Inductor. The talk will cover the state of the Torch-Spyre project, describe key functional enablement and performance improvements made in 2026 and contributions made back to PyTorch to enable such a tight integration.

Providing the expected PyTorch programming model and user experience on a dataflow accelerator while also achieving our performance goals was challenging. We will discuss key items such as enabling device-specific tensor layouts and scratchpad optimized tiling, with an emphasis on how our solutions integrate with upstream PyTorch. We hope this will be of interest to the community and enable other out-of-tree accelerators to more easily achieve deep PyTorch integration.

Enabling new hardware, supporting thousands of models, and scaling inference takes progress across the open-source AI stack. This keynote shares how IBM and Red Hat are contributing across the PyTorch ecosystem to make that happen. We’ll highlight work bringing IBM Spyre to the PyTorch compiler and vLLM, how AI tools are helping scale model enablement to thousands of models, and a look ahead at next-generation Spyre hardware. We’ll also cover Red Hat’s contributions to PyTorch and CI/CD infrastructure for out-of-tree accelerators, helping hardware backends keep pace with the community. Finally, we will touch on our journey with the llm-d community to enable efficient distributed inference at scale.

For many new contributors or adopters, the entry point into the PyTorch ecosystem is not any one resource; it’s the local community. The PyTorch Ambassador Program brings together rising community leaders from around the world to support them in activating, connecting, and growing interest in PyTorch at scale. The program was launched in 2025 by the PyTorch Foundation. This session offers a cross-country view of the PyTorch Ambassador Program, drawn from the ambassador activity reports spanning: what ambassadors built, the challenges they hit, and the results they produced. Beyond the numbers, it distills a reproducible model: recruit, educate, level up, then help local users into upstream contributors and examines how that model adapts to different contexts, from language and local ecosystem maturity to uneven access to compute. Attendees will gain a clear understanding of how to engage more deeply with the PyTorch community and position themselves for the Ambassador role through advocacy, education, and contribution. Whether you are an active contributor or someone looking to expand your role in the PyTorch community, this session offers actionable insights to help you take the next step to becoming a PyTorch Ambassador.

Recently, the PyTorch team released Helion, a new domain-specific PyTorch-based language for developing high-performing, portable kernels. With extensive autotuning and broader backend support, Helion promises to advance performance portability beyond Triton.

To test this, we took on the challenge of writing one of AI's most performance-critical kernels in Helion: Paged Attention, the core of vLLM.

Building on our expertise creating the state-of-the-art portable Triton attention backend in vLLM, we developed the experimental Helion attention backend for vLLM.

In this talk, we will present the differences between paged attention in Triton and Helion, what changes to the algorithm of the kernel were necessary to achieve matching performance, and how Helion enables optimizations that were not possible with Triton. Then, we will share lessons we learned along the way and extensive benchmark results that show that Helion kernels can, in many scenarios, even outperform our optimized Triton kernels. Our early results indicate the experimental Helion attention backend can reduce the latency by up to 50% and improve the end-to-end throughput vs. Triton by up to 10%.