Name entity recognition using language models

Abstract

This paper presents a new statistic name entity recognition algorithm, which does not require the collection and manual annotation of domain-specific sentences to train the models. The models of the name entities are domain-independent and could be directly applied to other domains of applications. This technique can also be applied to iteratively decode a set of raw sentences, if available, and use the decoded output to improve the statistic models. Applied to the mutual fund trading application, this new technique achieves a performance comparable to that using the decision tree model, which is trained from an annotated corpus. Iterative decoding of a set of natural language utterances and training of the general language model decreases the sentence error rate by 11%.