Recurrent Transformers Trade-off Parallelism for Length Generalization on Regular Languages
- Paul Soulos
- Aleksandar Terzic
- et al.
- 2024
- NeurIPS 2024
Research Interests:
My main research focuses on enhancing sample efficiency, enabling machine learning and reasoning methods to reliably generalize from as little data as possible. I am also interested in co-designing algorithms alongside emerging hardware technologies, with a strong emphasis on improving robustness, computational performance, and energy efficiency. Additionally, I am broadly interested in exploiting approximation opportunities across computation, communication, sensing, and storage systems.
Short Biography:
Abbas Rahimi received the B.S. degree in computer engineering from the University of Tehran in 2010, and the M.S. and Ph.D. degrees in computer science and engineering from the University of California San Diego in 2015, and subsequently was a postdoctoral fellow at the University of California Berkeley and the ETH Zürich. In 2020, he joined the IBM Research-Zürich laboratory as a Research Staff Member.
He has received the 2015 Outstanding Dissertation Award in the area of 'New Directions in Embedded System Design and Embedded Software'' from the European Design and Automation Association, and the ETH Zürich Postdoctoral Fellowship in 2017. He was a co-recipient of the Best Paper Nominations at DAC (2013) and DATE (2019), and the Best Paper Awards at BICT (2017), BioCAS (2018), and IBM's Pat Goldberg Memorial (2020).
Selected Publications:
J. Thomm, G. Camposampiero, A. Terzic, M. Hersche, B. Schölkopf, A. Rahimi, 'Limits of transformer language models on learning to compose algorithms', Conference on Neural Information Processing Systems (NeurIPS), 2024.
N. Menet, M. Hersche, K. Karunaratne, L. Benini, A. Sebastian, A. Rahimi, 'MIMONets: multiple-input-multiple-output neural networks exploiting computation in superposition' Conference on Neural Information Processing Systems (NeurIPS), 2023.
M. Hersche, M. Zeqiri, L. Benini, A. Sebastian, A. Rahimi, 'A neuro-vector-symbolic architecture for solving Raven’s progressive matrices', Nature Machine Intelligence, 2023.
J. Langenegger, G. Karunaratne, M. Hersche, L. Benini, A. Sebastian, A. Rahimi, 'In-memory factorization of holographic perceptual representations', Nature Nanotechnology, 2023.
M. Hersche, G. Karunaratne, G. Cherubini, L. Benini, A. Sebastian, A. Rahimi, 'Constrained few-shot class-incremental learning', Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
G. Karunaratne, M. Schmuck, M. Le Gallo, G. Cherubini, L. Benini, A. Sebastian, A. Rahimi, 'Robust high-dimensional memory-augmented neural networks', Nature Communications, 2021. (Featured in the 50 best articles in the Applied Physics and Mathematics)
A. Moin, A. Zhou, A. Rahimi, et al., 'A wearable biosensing system with in-sensor adaptive machine learning for hand gesture recognition', Nature Electronics, 2021.
G. Karunaratne, M. Le Gallo, G. Cherubini, L. Benini, A. Rahimi, A. Sebastian, 'In-memory hyperdimensional computing', Nature Electronics, 2020. (Appeared on the cover June issue 2020; Received IBM's Pat Goldberg Memorial Best Paper Awards)
A. Burrello, K. Schindler, L. Benini, A. Rahimi, 'Hyperdimensional computing with local binary patterns: one-shot learning of seizure onset and identification of ictogenic brain regions using short-time iEEG recordings,” IEEE Transactions on Biomedical Engineering (TBME), 2020.
A. Rahimi, P. Kanerva, L. Benini, J. M. Rabaey, 'Efficient biosignal processing using hyperdimensionalcomputing: network templates for combined learning and classification of ExG signals', Proceedings of the IEEE, 2018.
A. Rahimi, S. Datta, D. Kleyko, E. P. Frady, B. Olshausen, P. Kanerva, J. M. Rabaey, 'High-dimensional computing as a nanoscalable paradigm', IEEE Transactions on Circuits and Systems (TCAS-I), 2017.
A. Rahimi, P. Kanerva, J. M. Rabaey, 'A robust and energy-efficient classifier using brain-inspired hyperdimensional computing', International Symposium on Low Power Electronics and Design (ISLPED), 2016.
Research in the News:
A New Approach to Computation Reimagines Artificial Intelligence
Disentangling visual concepts by embracing stochastic in-memory computing
The best of both worlds: Deep learning meets vector-symbolic architectures
High-five or thumbs-up? New device detects which hand gesture you want to make
Fulfilling Brain-inspired Hyperdimensional Computing with In-memory Computing