Dmitry Krotov

I am a physicist working on neural networks and machine learning. I am a member of the research staff at the MIT-IBM Watson AI Lab and IBM Research in Cambridge, MA. Prior to this, I was a member of the Institute for Advanced Study in Princeton. Broadly defined, my research focuses on the computational properties of neural networks. Particularly, I am interested in implementing ideas coming from neuroscience and physics in modern AI systems. I received a PhD in Physics from Princeton University in 2014. If you want to learn more about our work please check out this recent Q&A with me.

Email: krotov@ibm.com

Publications, see also Google Scholar

2024

• L.Kozachkov, JJ.Slotine, D.Krotov, Neuron-Astrocyte Associative Memory, COSYNE 2024.
• Z.He, L.Karlinsky, D.Kim, J.McAuley, D.Krotov, R.Feris, CAMELoT: Towards Large Language Models with Training-Free Consolidated Associative Memory.

2023

• D.Krotov, A new frontier for Hopfield networks, Nature Reviews Physics, free pdf.
• L.Kozachkov, K.Kastanenka, D.Krotov, Building transformers from neurons and astrocytes, PNAS.
• B.Hoover, Y.Liang, B.Pham, R.Panda, H.Strobelt, D.H.Chau, M.J.Zaki, D.Krotov, Energy transformer, NeurIPS 2023, video.
• H.Chaudhry, J.Zavatone-Veth, D.Krotov, C.Pehlevan, Long Sequence Hopfield Memory, NeurIPS 2023, video.
• B.Saha, D.Krotov, M.J.Zaki, P.Ram, End-to-end Differentiable Clustering with Associative Memories, ICML 2023.
• T.Bricken, X.Davies, D.Singh, D.Krotov, G.Kreiman, Sparse Distributed Memory is a Continual Learner, ICLR 2023.
• B.Hoover, H.Strobelt, D.Krotov, J.Hoffman, Z.Kira, D.H.Chau, Memory in Plain Sight: A Survey of the Uncanny Resemblances between Diffusion Models and Associative Memories.
• D.Tyulmankov, K.Stachenfeld, D.Krotov, L.Abbott, Memorization and consolidation in associative memory networks, Associative Memory & Hopfield Networks in 2023, NeurIPS 2023.
• D.Krotov, Modern Hopfield Networks in AI and Neurobiology, American Physical Society March Meeting 2023.

2022

• B.Hoover, D.H.Chau, H.Strobelt, D.Krotov, A Universal Abstraction for Hierarchical Hopfield Networks, Symbiosis of Deep Learning and Differential Equations workshop, NeurIPS 2022.
• Y.Liang, D.Krotov, M.J.Zaki, Modern Hopfield Networks for graph embedding, Frontiers in Big Data.

2021

• D.Krotov, J.Hopfield, Large associative memory problem in neurobiology and machine learning, ICLR 2021.
• Y.Liang, C.K.Ryali, B.Hoover, L.Grinberg, S.Navlakha, M.J.Zaki, D.Krotov, Can a Fruit Fly Learn Word Embeddings?, ICLR 2021.
• D.Krotov, Hierarchical associative memory.

2020

• C.K.Ryali, J.Hopfield, L.Grinberg, D.Krotov, Bio-Inspired Hashing for Unsupervised Similarity Search, ICML 2020.

2019

• D.Krotov, J.Hopfield, Unsupervised learning by competing hidden units, PNAS.
• L.Grinberg, J.Hopfield, D.Krotov, Local Unsupervised Learning for Image Analysis, Real Neurons & Hidden Units workshop, NeurIPS 2019.

2018

• D.Krotov, J.Hopfield, Dense associative memory is robust to adversarial inputs, Neural Computation.

2017

• D.Krotov, J.Hopfield, Feature to prototype transition in neural networks, American Physical Society March Meeting 2017.

2016

• D.Krotov, J.Hopfield, Dense associative memory for pattern recognition, NeurIPS 2016.

2014

• D.Krotov, J.O.Dubuis, T.Gregor, W.Bialek, Morphogenesis at criticality, PNAS.
• D.Krotov, Strong correlations in gravity and biophysics, PhD Thesis, Princeton University 2014.

2011

• D.Krotov, A.M.Polyakov, Infrared sensitivity of unstable vacua, Nuclear Physics B.

Selected Talks

• Dense Associative Memory for Pattern Recognition, NeurIPS 2016 oral talk.
• Associative Memory & Hopfield Networks in 2023, NeurIPS 2023.
• Harvard Center Of Mathematical Sciences And Applications, 2023.
• MIT Center for Brains, Minds, and Machines summer school, Woods Hole 2023.
• Deep Learning from the Perspective of Physics and Neuroscience, KITP 2023.
• Lecture at Introduction to Deep Learning, MIT course 6.S191, 2019.
• Fireside chat between Sepp Hochreiter and Dmitry Krotov, NeurIPS 2020.
• Seminar at Microsoft Research, 2018.
• Institute of Advanced Research in Artificial Intelligence, 2020.
• Foundations of Machine Learning and Its Applications for Scientific Discovery in Physical and Biological Systems, NSF, Washington DC, 2022.

Our work in the news

• Q&A, In search of AI algorithms that mimic the brain.
• MIT News, AI models are powerful, but are they biologically plausible?
• Decrypt, Scientists Uncover Biological Echoes in Powerful AI Transformer Models.
• Quanta Magazine, How Transformers Seem to Mimic Parts of the Brain.
• IBM Research Blog, AI transformers shed light on the brain’s mysterious astrocytes.
• TechCrunch, This week in AI: Amazon ‘enhances’ reviews with AI while Snap’s goes rogue.
• The New Indian Express, Making AI systems biologically plausible.
• InfoQ, Transformers Can Mock Part of Human Brain.
• Discover Magazine, Fruit Fly Brain Hacked For Language Processing.
• VentureBeat, IBM’s biology-inspired AI generates better hash codes than classical approaches.
• Neurohive, IBM Researchers Proposed New Biologically Plausible Learning Algorithm for Neural Networks.
• Datanami, Toward Biologically Plausible AI.