TY - CHAP A1 - Wegmann, Steven A1 - Faria, Arlo A1 - Janin, Adam A1 - Riedhammer, Korbinian A1 - Morgan, Nelson T1 - The Tao of ATWV: Probing the mysteries of keyword search performance T2 - IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU), Olomouc, Czech Republic, Dezember 2013. N2 - In this paper we apply diagnostic analysis to gain a deeper understanding of the performance of the the keyword search system that we have developed for conversational telephone speech in the IARPA Babel program. We summarize the Babel task, its primary performance metric, “actual term weighted value” (ATWV), and our recognition and keyword search systems. Our analysis uses two new oracle ATWV measures, a bootstrap-based ATWV confidence interval, and includes a study of the underpinnings of the large ATWV gains due to system combination. This analysis quantifies the potential ATWV gains from improving the number of true hits and the overall quality of the detection scores in our system's posting lists. It also shows that system combination improves our systems' ATWV via a small increase in the number of true hits in the posting lists. KW - Speech Recognition Y1 - 2013 ER -