Structured Sparse Transition Matrices to Enable State Tracking in State-Space Models
- Aleksandar Terzic
- Nicolas Menet
- et al.
- 2025
- NeurIPS 2025
Research Interests:
My primary research interest lies in creating formally expressive machine learning and reasoning algorithms, yet with scalable and high-performance implementation on classical and quantum computing platforms. This includes AI agents based on hybrid foundation models, with the main emphasis on improving the quality of AI-driven design and discovery. Additionally, I am broadly interested in exploiting cross-layer approximation opportunities spanning computation, communication, sensing, and storage systems to significantly reduce computational complexity and energy consumption.
Short Biography:
I received my M.S. and Ph.D. in Computer Science and Engineering from UC San Diego in 2015. I was a postdoctoral fellow at UC Berkeley and ETH Zürich before joining IBM Research–Zürich in 2020. I am currently a Research Staff Member in the Mathematics of Computation Department.
I have received the 2015 Outstanding Dissertation Award in the area of "New Directions in Embedded System Design and Embedded Software" from the European Design and Automation Association, and the ETH Zürich Postdoctoral Fellowship in 2017. I was a co-recipient of the Best Paper Nominations at DAC (2013) and DATE (2019), and the Best Paper Awards at BICT (2017), BioCAS (2018), and IBM's Pat Goldberg Memorial (2020).
Selected Publications:
A. Terzic, N. Menet, M. Hersche, T. Hofmann, A. Rahimi, Structured sparse transition matrices to enable state tracking in state-space models, Conference on Neural Information Processing Systems (NeurIPS), 2025. (Spotlight paper)
G. Camposampiero, P. Barbiero, M. Hersche, R. Wattenhofer, A. Rahimi, Scalable evaluation and neural models for compositional generalization, Conference on Neural Information Processing Systems (NeurIPS), 2025.
J. Büchel, I. Chalas, G. Acampa, A. Chen, O. Fagbohungbe, S. Tsai, K. El Maghraoui, M.Le Gallo, A. Rahimi, A. Sebastian, Analog foundation models, Conference on Neural Information Processing Systems (NeurIPS), 2025.
F. Carzaniga, G. Hoppeler, M. Hersche, K. Schindler, A. Rahimi, The case for cleaner biosignals: high-fidelity neural compressor enables transfer from cleaner iEEG to noisier EEG, International Conference on Learning Representations (ICLR), 2025.
A. Terzic, M. Hersche, G. Camposampiero, T. Hofmann, A. Sebastian, A. Rahimi, On the expressiveness and length generalization of selective state space models on regular languages, AAAI Conference on Artificial Intelligence, 2025.
J. Büchel, A. Vasilopoulos, W. Simon, I. Boybat, H. Tsai, G. Burr, H. Castro, B. Filipiak, M. Le Gallo, A. Rahimi, V. Narayanan, A. Sebastian, Efficient scaling of large language models with mixture of experts and 3D analog in-memory computing, Nature Computational Science, 2025. (Featured on the cover Jan issue 2025. Highlighted in Nat. Comput. Sci's News & Views)
J. Thomm, G. Camposampiero, A. Terzic, M. Hersche, B. Schölkopf, A. Rahimi, Limits of transformer language models on learning to compose algorithms, Conference on Neural Information Processing Systems (NeurIPS), 2024.
J. Büchel, G. Camposampiero, A. Vasilopoulos, C. Lammie, M. Le Gallo, A. Rahimi, A. Sebastian, Kernel approximation using analog in-memory computing, Nature Machine Intelligence, 2024.
N. Menet, M. Hersche, K. Karunaratne, L. Benini, A. Sebastian, A. Rahimi, MIMONets: multiple-input-multiple-output neural networks exploiting computation in superposition Conference on Neural Information Processing Systems (NeurIPS), 2023.
M. Hersche, M. Zeqiri, L. Benini, A. Sebastian, A. Rahimi, A neuro-vector-symbolic architecture for solving Raven’s progressive matrices, Nature Machine Intelligence, 2023. (Spotlighted by Quanta Magazine as one of 2023's biggest breakthroughs in CS)
J. Langenegger, G. Karunaratne, M. Hersche, L. Benini, A. Sebastian, A. Rahimi, In-memory factorization of holographic perceptual representations, Nature Nanotechnology, 2023. (Highlighted in Nat. Nanotechnol.'s News & Views)
M. Hersche, G. Karunaratne, G. Cherubini, L. Benini, A. Sebastian, A. Rahimi, Constrained few-shot class-incremental learning, Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
G. Karunaratne, M. Schmuck, M. Le Gallo, G. Cherubini, L. Benini, A. Sebastian, A. Rahimi, Robust high-dimensional memory-augmented neural networks, Nature Communications, 2021. (Featured in the 50 best articles in the Nat. Comm's Applied Physics and Mathematics)
A. Moin, A. Zhou, A. Rahimi, et al., A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition, Nature Electronics, 2021. (Highlighted in Nat. Electron.'s News & Views)
G. Karunaratne, M. Le Gallo, G. Cherubini, L. Benini, A. Rahimi, A. Sebastian, In-memory hyperdimensional computing, Nature Electronics, 2020. (Featured on the cover June issue 2020. Received IBM's Pat Goldberg Memorial Best Paper Awards)
T. Wu, H. Li, P-C. Huang, A. Rahimi, J. M. Rabaey, H-S. Wong, M. Shulaker, S. Mitra, Brain-inspired computing exploiting carbon nanotube FETs and resistive RAM: Hyperdimensional computing case study, IEEE International Solid-State Circuits Conference (ISSCC), 2018.
A. Rahimi, S. Datta, D. Kleyko, E. P. Frady, B. Olshausen, P. Kanerva, J. M. Rabaey, High-dimensional computing as a nanoscalable paradigm, IEEE Transactions on Circuits and Systems (TCAS-I), 2017.
A. Rahimi, P. Kanerva, J. M. Rabaey, A robust and energy-efficient classifier using brain-inspired hyperdimensional computing, International Symposium on Low Power Electronics and Design (ISLPED), 2016.
Research in the News:
A New Approach to Computation Reimagines Artificial Intelligence
Disentangling visual concepts by embracing stochastic in-memory computing
The best of both worlds: Deep learning meets vector-symbolic architectures
High-five or thumbs-up? New device detects which hand gesture you want to make
Fulfilling Brain-inspired Hyperdimensional Computing with In-memory Computing