Audio-visual ASR from multiple views inside smart rooms
Gerasimos Potamianos, Patrick Lucey
MFI 2006
This paper addresses the problem of audio-visual information fusion to provide highly robust speech recognition. We investigate methods that make different assumptions about asynchrony and conditional dependence across streams and propose a technique based on composite HMMs that can account for stream asynchrony and different levels of information integration. We show how these models can be trained jointly based on maximum likelihood estimation. Experiments, performed for a speaker-independent large vocabulary continuous speech recognition task and different integration methods, show that best performance is obtained by asynchronous stream integration. This system reduces the error rate at a 8.5 dB SNR with additive speech "babble" noise by 27% relative over audio-only models and by 12% relative over traditional audio-visual models using concatenative feature fusion.
Gerasimos Potamianos, Patrick Lucey
MFI 2006
Shahram Ebadollahi, James W. Cooper, et al.
CEUR-WS 2008
Gerasimos Potamianos, Juergen Luettin, et al.
ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Vit Libal, Jonathan Connell, et al.
MMSP 2007