TOWARDS SPEECH UNDERSTANDING ACROSS MULTIPLE LANGUAGES

Abstract

In this paper we describe our initial efforts in building a natural language understanding (NLU) system across multiple languages. The system allows users to switch languages seamlessly in a single session without requiring any switch in the speech recognition system. Context dependence is maintained across sentences, even when the user changes languages. Towards this end we have begun building a universal speech recognizer for English and French languages. We experiment with a universal phonology for both French and English with a novel mechanism to handle language dependent variations. Our best results so far show about 5% relative performance degradation for English relative to a unilingual English system and a 9% relative degradation in French relative to a unilingual French system. The NLU system uses the same statistical understanding algorithms for each language, making system development, maintenance, and portability vastly superior to systems built customly for each language.