Symbol sequence search from telephone conversation

Abstract

We propose a method for searching for symbol sequences in conversations. Symbol sequences can include phone numbers, credit card numbers, and any kind of ticket (identification) numbers and are often communicated in call center conversations. Automatic extraction of these from speech is a key to many automatic speech recognition (ASR) applications such as question answering and summarization. Compared with spoken term detection (STD), symbol sequence searches have two additional problems. First, the entire symbol sequence is typically not observed continuously but in sub sequences, where customers or agents speak these sequences in fragments, while the recipient repeats them to ensure they have the correct sequence. Second, we have to distinguish between different symbol sequences, for example, phone numbers versus ticket numbers or customer identification numbers. To deal with these problems, we propose to apply STD to symbol-sequence fragments and subsequently use confidence scoring to obtain the entire symbol sequence. For the confidence scoring, We propose a long short-term memory (LSTM) based approach that inputs word before and after fragments. We also propose to detect repetitions of fragments and use it for confidence scoring. Our proposed method achieves a 0.87 F-measure, in an eight-digit customer identification number search task, when operating at 20.3% WER.