Unsupervised deep belief features for speech translation

Abstract

We present a novel formalism for introducing deep belief features to Hierarchical Machine Translation Model. The deep features are generated by unsupervised training of a deep belief network built with stacked sets of Restricted Boltzmann Machines. We show that our new deep feature based hierarchical model is better than the baseline hierarchical model with gains for two different languages pairs in two different data size settings. We obtain absolute BLEU score improvement of +1.13 on Darito-English and +0.66 on English-to-Dari Transtac Evaluation task. We also observe gains on English-to-Chinese translation task.