Audio-visual speech enhancement with AVCDCN (Audio-Visual Codebook Dependent Cepstral Normalization)

Abstract

In this paper, we introduce a non-linear enhancement technique called Audio-visual Codebook Dependent Cepstral Normalization (AVCDCN) and we consider its use with both audio-only and audio-visual speech recognition. AVCDCN is inspired from CDCN [ 11 [2], an audio-only enhancement technique that approximates the non-linear effect of noise on speech with a piece-wise constant function. Our experiments show that the use of visual information in AVCDCN allows significant performance gains over CDCN.