Large-Scale Speaker Ranking from Crowdsourced Pairwise Listener Ratings
- Speech quality and likability is a multi-faceted phenomenon consisting of a combination of perceptory features that cannot easily be computed nor weighed automatically. Yet, it is often easy to decide which of two voices one likes better, even though it would be hard to describe why, or to name the underlying basic perceptory features. Although likability is inherently subjective and individual preferences differ frequently, generalizations are useful and there is often a broad intersubjective consensus about whether one speaker is more likable than another. However, breaking down likability rankings into pairwise comparisons leads to a quadratic explosion of rating pairs. We present a methodology and software to efficiently create a likability ranking for many speakers from crowdsourced pairwise likability ratings. We collected pairwise likability ratings for many (>220) speakers from many raters (>160) and turn these ratings into one likability ranking. We investigate the resulting speaker ranking stability under different conditions: limiting the number of ratings and the dependence on rater and speaker characteristics. We also analyze the ranking wrt. acoustic correlates to find out what factors influence likability. We publish our ranking and the underlying ratings in order to facilitate further research.
Author: | Timo BaumannORCiDGND |
---|---|
DOI: | https://doi.org/10.21437/Interspeech.2017-1697 |
Parent Title (English): | Interspeech 2017 : 20-24 August 2017 |
Publisher: | ISCA |
Document Type: | conference proceeding (article) |
Language: | English |
Year of first Publication: | 2017 |
Release Date: | 2022/06/24 |
Tag: | crowd-sourcing; found data; likability; ranking; ratings; speech quality |
First Page: | 2262 |
Last Page: | 2266 |
Institutes: | Fakultät Informatik und Mathematik |
Begutachtungsstatus: | peer-reviewed |
Publication: | Externe Publikationen |
research focus: | Information und Kommunikation |
Licence (German): | Keine Lizenz - Es gilt das deutsche Urheberrecht: § 53 UrhG |