Thinking Fast and Slow in AI: the Role of Metacognition

Abstract

AI systems have seen dramatic advancement in recent years, bringing many applications that pervade our everyday life. However, we are still mostly seeing instances of narrow AI: many of these recent developments are typically focused on a very limited set of competencies and goals, e.g., image interpretation, natural language processing, classification, prediction, and many others. Moreover, while these successes can be accredited to improved algorithms and techniques, they are also tightly linked to the availability of huge datasets and computational power. State-of-the-art AI still lacks many capabilities that would naturally be included in a notion of (human) intelligence. Examples of these capabilities are generalizability, adaptability, robustness, explainability, causal analysis, abstraction, common sense reasoning, ethical reasoning, as well as a complex and seamless integration of learning and reasoning supported by both implicit and explicit knowledge. We argue that a better study of the mechanisms that allow humans to have these capabilities can help us understand how to imbue AI systems with these competencies. We focus especially on D. Kahneman’s theory of thinking fast and slow, and we propose a multi-agent AI architecture (called SOFAI, for SlOw and Fast AI) where incoming problems are solved by either system 1 (or "fast") agents (also called "solvers"), that react by exploiting only past experience, or by system 2 (or "slow") agents, that are deliberately activated when there is the need to reason and search for optimal solutions beyond what is expected from the system 1 agent. Both kinds of agents are supported by a model of the world, containing domain knowledge about the environment, and a model of “self”, containing information about past actions of the system and solvers’ skills. Given the need to choose between these two kinds of solvers, a meta-cognitive agent is employed, performing introspection and arbitration roles, and assessing the need to employ system 2 solvers by considering resource constraints, abilities of the solvers, past experience, and expected reward for a correct solution of the given problem. To do this balancing in a resource-conscious way, the meta-cognitive agent includes two successive phases, the first one faster and more approximate, and the second one (if needed) more careful and deliberate. Different approaches to the design of AI systems inspired by the dual-system theory have also been published recently. Many real-life settings present sequential decision problems. Depending on the availability of system 1 and/or system 2 solvers that can tackle single decisions or a sequence of them, the SOFAI architecture employs the meta-cognitive agent at each decision, or only once for a whole sequence. The first modality provides additional flexibility, since each call of the meta-cognitive module may choose a different solver to make the next decision, while the second one allows to exploit additional domain knowledge in the solvers.