Computing Team Process Measures from the Structure and Content of Broadcast Collaborative Communications

Abstract

Existing approaches to compute team process measures are primarily based on survey ratings, semantic classification of communications, and social network analyses. Although existing approaches reveal important information about team performance, they face specific limitations. Survey methodologies are in general unreliable, biased, and not dynamic; communication classifications are often a-theoretical; and social network analytics ignore the meanings of messages. Accordingly, we develop a better-defined formal empirical approach for computing team process measures. Our contribution builds on existing work in semantic classification of messages in broadcast communications and proposes a general set of meanings of messages for team processes. Using the meanings of messages, we propose formal approaches to compute team process measures. We evaluate these measures using a military data set and find the following. First, our text mining approach to infer meanings of messages significantly improves over the bag of words approach and yields macroaverage and microaverage F-measures of 70% and 80%, respectively. Second, compared with baseline measures such as degree centrality, cognitive processes remain significantly stable with time, whereas measures such as affective process significantly increase with time.