Word level confidence measurement using semantic features

Abstract

This paper proposes two principled methods to incorporate semantic information into word level confidence measurement. The first technique uses tag and arc probabilities obtained from a statistical classer and parser tree. The second technique uses a maximum entropy based semantic structured language model to use semantic structure of a sentence to assign semantic probabilities to each word. Semantic features provide significant improvements over a posterior probability based confidence measure when used together in an air travel reservation task.