Estimating class membership probabilities using classifier learners

Abstract

We present an algorithm, "Probing", which reduces learning an estimator of class probability membership to learning binary classi fiers. The reduction comes with a theoreti cal guarantee: a small error rate for binary classification implies accurate estimation of class membership probabilities. We tested our reduction on several datasets with several classifier learning algorithms. The results show strong performance as compared to other common methods for obtaining class membership probability estimates from clas siers.