TY - CHAP A1 - Riedhammer, Korbinian A1 - Gillick, Dan A1 - Favre, Benoit A1 - Hakkani-Tür, Dilek T1 - Packing the Meeting Summarization Knapsack T2 - INTERSPEECH 2008, 9th Annual Conference of the International Speech Communication Association (ISCA), Brisbane, Australia, September 2008. N2 - Despite considerable work in automatic meeting summarization over the last few years, comparing results remains difficult due to varied task conditions and evaluations. To address this issue, we present a method for determining the best possible extractive summary given an evaluation metric like ROUGE. Our oracle system is based on a knapsack-packing framework, and though NP-Hard, can be solved nearly optimally by a genetic algorithm. To frame new research results in a meaningful context, we suggest presenting our oracle results alongside two simple baselines. We show oracle and baseline results for a variety of evaluation scenarios that have recently appeared in this field. KW - summarization KW - meetings KW - evaluation Y1 - 2008 SP - 2434 EP - 2437 ER - TY - CHAP A1 - Gillick, Dan A1 - Riedhammer, Korbinian A1 - Favre, Benoit A1 - Hakkani-Tür, Dilek T1 - A Global Optimization Framework for Meeting Summarization T2 - 2009 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Taipei, Taiwan, April 2009. N2 - We introduce a model for extractive meeting summarization based on the hypothesis that utterances convey bits of information, or concepts. Using keyphrases as concepts weighted by frequency, and an integer linear program to determine the best set of utterances, that is, covering as many concepts as possible while satisfying a length constraint, we achieve ROUGE scores at least as good as a ROUGE-based oracle derived from human summaries. This brings us to a critical discussion of ROUGE and the future of extractive meeting summarization. KW - meeting summarization KW - integer linear programming KW - summarization evaluation Y1 - 2009 SP - 4769 EP - 4772 ER -