The Design and Development of Games with a Purpose for AI Systems

Abstract

Collecting training data is often time-consuming, expensive and imposes a bottleneck on many machine learning tasks. Much of training data used to train ML systems is a result of the work of crowdworkers who are paid to do routinizable mundane tasks. Games with a purpose leverage game mechanics to use the perceptive capacities of users to collect data in a way that is more enjoyable to crowdworkers. Different machine learning tasks require different types of training data. In this paper, we discuss the design and development of building two games with a purpose: Guess the Word and Fool the AI, designed to collect data from both crowdworkers and domain experts for two very different machine learning problems. To make these games enjoyable and interactive, a team of engineers, research scientists and designers create new games with a purpose around various machine learning tasks. In this paper, we describe the design of these games, how we incorporate game mechanics within these games to make the collection of annotation tasks more efficient but also enjoyable.