A semantic approach to the dynamic design of interaction controls in conversation systems

Abstract

To support a full-fledged, multimedia human-computer conversation, we are building an intelligent framework, called Responsive Information Architect (RIA), which can automatically synthesize multimedia responses during the conversation. As part of its visual response generation, RIA dynamically creates context-sensitive interaction controls that are visual interfaces through which users can further interact with RIA. To enable the systematic design of interaction controls, we study and abstract control properties. In particular, in this paper we present a semantic model that captures the intentional, presentational, and behavioral characteristics of interaction controls. Using this model, we show how to systematically sketch the semantics of an interaction control.