User ratings of ontologies: Who will rate the raters?

Abstract

The number of ontologies and knowledge bases covering different domains and available on the World-Wide Web is steadily growing. As more ontologies are available, it is becoming harder, and not easier, for users to find ontologies they need. How do they evaluate if a particular ontology is appropriate for their task? How do they choose among many ontologies for the same domain? We argue that allowing users on the Web to annotate and review ontologies is an important step in facilitating ontology evaluation and reuse for others. However, opening the system to everyone on the Web poses a problem of trust: Users must be able to identify reviews and annotations that are useful for them. We discuss the kinds of metadata that we can collect from users and authors of ontologies in the form of annotations and reviews, explore the use of an Open Rating System for evaluating ontologies and knowledge sources, and present a brief overview of a Web-based browser for Protégé ontologies that enables users to annotate information in ontologies. Copyright © 2005, American Association for Artificial Intelligence (www.aaai.org). All rights reserved.