About cookies on this site Our websites require some cookies to function properly (required). In addition, other cookies may be used with your consent to analyze site usage, improve the user experience and for advertising. For more information, please review your options. By visiting our website, you agree to our processing of information as described in IBM’sprivacy statement. To provide a smooth navigation, your cookie preferences will be shared across the IBM web domains listed here.
Publication
INTERSPEECH 2017
Conference paper
Efficient emotion recognition from speech using deep learning on spectrograms
Abstract
We present a new implementation of emotion recognition from the para-lingual information in the speech, based on a deep neural network, applied directly to spectrograms. This new method achieves higher recognition accuracy compared to previously published results, while also limiting the latency. It processes the speech input in smaller segments - up to 3 seconds, and splits a longer input into non-overlapping parts to reduce the prediction latency. The deep network comprises common neural network tools - convolutional and recurrent networks - which are shown to effectively learn the information that represents emotions directly from spectrograms. Convolution-only lower-complexity deep network achieves a prediction accuracy of 66% over four emotions (tested on IEMOCAP - a common evaluation corpus), while a combined convolution-LSTM higher-complexity model achieves 68%. The use of spectrograms in the role of speech-representing features enables effective handling of background non-speech signals such as music (excl. singing) and crowd noise, even at noise levels comparable with the speech signal levels. Using harmonic modeling to remove non-speech components from the spectrogram, we demonstrate significant improvement of the emotion recognition accuracy in the presence of unknown background non-speech signals.