Sygma: System for generalizable modular question answering over knowledge bases

Abstract

Knowledge Base Question Answering (KBQA) tasks that in-volve complex reasoning are emerging as an important re-search direction. However, most KBQA systems struggle withgeneralizability, particularly on two dimensions: (a) acrossmultiple reasoning types where both datasets and systems haveprimarily focused on multi-hop reasoning, and (b) across mul-tiple knowledge bases, where KBQA approaches are specif-ically tuned to a single knowledge base. In this paper, we present SYGMA, a modular approach facilitating generalizability across multiple knowledge bases and multiple reasoning types. Specifically, SYGMA contains three high level modules: 1) KB-agnostic question understanding module that is common across KBs 2) Rules to support additional reasoning types and 3) KB-specific question mapping and answering module to address the KB-specific aspects of the answer ex-traction. We demonstrate effectiveness of our system by evaluating on datasets belonging to two distinct knowledge bases, DBpedia and Wikidata. In addition, to demonstrate extensibility to additional reasoning types we evaluate on multi-hop reasoning datasets and a new Temporal KBQA benchmark dataset on Wikidata, namedTempQA-WD1, introduced in this paper. We show that our generalizable approach has better competetive performance on multiple datasets on DBpedia and Wikidata that requires both multi-hop and temporal reasoning