This AI could likely beat you at an IQ test

If you’ve ever taken an IQ test, you have likely come across Raven’s progressive matrices. Frequently used to measure nonverbal cognitive abilities, the test features an arrangement of several images in a matrix of rows and columns. The test taker is asked to select the right image to complete the matrix out of a set of possible answers. British psychologist John C. Raven, who came up with these puzzles in the 1930s, likely never imagined that machines would be trying to solve them almost a century later. But that’s just what we did.

In a paper published today in Nature Machine Intelligence, we reported on our recent research building a neuro-vector-symbolic AI model capable of solving Raven’s matrices with an 88% success rate. That figure surpasses the marks achieved by state-of-the-art deep learning as well as neuro-symbolic AI by at least 4%. That also happens to be a better score than the average human test taker.

The significance of this research lies not so much in the new record, however, but more in the demonstration of a novel way to combine traditional deep neural networks and symbolic AI more efficiently. We have paired the two together to create a sort of common language that allows the two approaches to communicate more smoothly than ever before.

Neuro-symbolic AI approaches aren’t new: Researchers have long tried to combine the superb perception capabilities of neural networks (such as the ability to recognize objects in an image) with the reasoning  ability (like identifying abstract relationships between objects such as their shape, size or color) provided by symbolic AI. But so far, neuro-symbolic AI has inherited the limitations of their individual deep learning and symbolic AI components. In neural networks, those limitations mainly come down to what researchers dub the binding problem: the inability to decompose joint representations to obtain distinct objects. For example, given a blue circle, a neural network is unable to separately recognize the properties “blue” and “circle.” So when it’s next presented with a red circle, the neural network can’t tell that it differs from the blue one by its color. Similarly, it can’t pinpoint the difference between, say, a blue circle and a blue square. Symbolic AI, on the other hand, can recognize those attributes but does so at the expense of computationally expensive database searches.

To address these two limitations, we propose a neuro-vector-symbolic architecture (NVSA) that combines neural networks and vector symbolic architecture (VSA) machinery in two crucial steps. The first important modification deals with the perception frontend in which the neural network operates to recognize objects encoding them in high-dimensional vectors. The key modification here is to introduce additional structure into those vectors to enable them to separate properties such as color or size from unstructured data. This is done through what we call a codebook.

Previous neuro-symbolic approaches attempt to bypass the binding problem by employing multiple complementary neural networks to unambiguously extract the item attributes from multiple objects in an image. This increases the number of weights and, as a result, the computational cost. Our NVSA eliminates that problem, while at the same time laying the foundation to boost the efficiency of the reasoning backend module.

With the NVSA in place we can then exploit specific mathematical operations that the VSA puts at our disposal to dramatically speed up the reasoning process.

In order to solve a Raven test, something called probabilistic abduction is required. This involves searching for a solution in a space defined by prior background knowledge. The prior knowledge is represented in symbolic form by describing all possible rule realizations that could govern the Raven tests. The purely symbolic reasoning approach needs to go through all valid combinations, compute the rule probability, and sum them up. This search becomes a computational bottleneck in the large search space, due to a large number of combinations.

Using our NVSA, however, we are able to perform those probabilistic calculations in a single vector operation. This is why our NVSA approach can solve abstract reasoning and analogy problems such as those present in Raven’s progressive matrices faster and more accurately than the state-of-the-art deep neural network and neuro-symbolic approaches.

We have been working on improving various aspects of deep learning perception by exploiting the VSA machinery since 2020. This has led to state-of-the-art accuracy in the largest few-shot learning tasks, as well as continual learning tasks on various image datasets. With this latest paper, we extend its application from perception to reasoning, to perform probabilistic reasoning with less expensive operations on the distributed representations of NVSA.

NVSA is the new state-of-the-art method for solving IQ tests

NVSA outperforms the state-of-the-art deep neural network and neuro-symbolic approaches. On the I-RAVEN dataset, it achieves a new accuracy record of 88.1%. This is at least 4.2% higher than the deep neural networks, and 17% higher than the state-of-the-art neuro-symbolic approach. In addition, NVSA reasoning through search in superposition is two orders of magnitude faster than the symbolic reasoning component within the traditional neuro-symbolic approaches.

Neuro-symbolic approaches typically rely on the fact that different components of a cognitive system can be effectively separated into components that are symbolic and into components that are statistical. NVSA provides an efficient means to perform expensive symbolic computations on less expensive and scalable distributed representations and operations. This paves the way to leveraging the power of structured computation and structured knowledge while taking advantage of distributed representations, potentially leading us down a path toward systems that as computationally efficient as our own brains.

Building upon this work, we are now planning to tackle the more complex task of generalization to various unseen combinations as the next challenge for NVSA.