The Text Analysis Conference (TAC) ranks summarization systems by their average score over a collection of document sets. We investigate the statistical appropriateness of this score and propose an alternative that better distinguishes between human and machine evaluation systems.
['Peter A. Rankel', 'John M. Conroy', 'Eric V. Slud', "Dianne P. O'Leary"]
Ranking Human and Machine Summarization Systems