Power mean pyramid scores for summarization evaluation

Abstract

We present Power Mean Pyramid Scores (PMP), an evaluation metric that extends the Pyramid evaluation scheme for summarization by combining Sentence Content Units (SCU) scores using Power Mean. The Pyramid method generates a summarization score by linearly combining component SCU scores. We find that by combining SCU scores using Power Mean, we can optimize a single parameter, a, leading to significantly improved correlation with human judgements. We demonstrate this result through an empirical study based on TAC-08 evaluation.