What makes a Model Compositional? A Neuro-Symbolic Theoretical View
- Parikshit Ram
- Tim Klinger
- et al.
- 2024
- IJCAI 2024
I am a senior research scientist in the Theoretical Mathematics and Computer Science (TMCS) department at IBM working in the area of machine learning and AI reasoning. My focus is on machine learning algorithms which generalize systematically (like computer programs with abstract variables) when possible. Current AI models do poorly when systematic generalization is required (for example in planning or reasoning) or else require a vast amount of data. On the other hand, some tasks are difficult to compress into small programs (e.g. continuous control) and less systematic approaches such as large Transformer models are required. We are looking at techniques which "right-size" the model complexity to the task, capturing as much regularity as possible using small resource-constrained programs, while allowing more flexible, larger modules, like LLM Transformers to handle everything else.
Our work is related to compositional generalization and compositional learning theory, program induction, algorithmic complexity, modularity, neuro-symbolic learning, circuit complexity, discrete search, and theorem proving.
My education was at McGill University in Canada (Joint Honors BA in Mathematics and Computer Science) and then at NYU (MS and PhD degrees in Computer Science).