Evaluation of real-time captioning by machine recognition with human support

Abstract

Verbal meetings are important at work, but employees who are deaf or hard of hearing (DHH) find it difficult to participate. Manual real-time captioning is a solution, but professional stenographers are too expensive for routine use. We are exploring the possibilities of real-time captioning that combines Automated Speech Recognition (ASR) and human capabilities, which can dramatically decrease these costs and thus improve the lives of DHH employees. We developed a flexible ASR-based real-time captioning tool that can be used by non-expert captioners to correct the recognized text in practical workplace situations. In this paper, we will report on our early results, focusing on accuracy and latency.