TY - CHAP A1 - Baumann, Timo A1 - Saboo, Ashutosh ED - Ehrmann, Maud T1 - Evaluating Heuristics for Audio-Visual Translation T2 - Proceedings of the Conference of Computational Humanities Research, (CHR2021): November 17-19, 2021, Amsterdam, The Netherlands N2 - Dubbing, i.e., the lip-synchronous translation and revoicing of audio-visual media into a target language from a different source language, is essential for the full-fledged reception of foreign audio-visual media, be it movies, instructional videos or short social media clips. In this paper, we objectify influences on the ‘dubbability’ of translations, i.e., how well a translation would be synchronously revoiceable to the lips on screen. We explore the value of traditional heuristics used in evaluating the qualitative aspects, in particular matching bilabial consonants and the jaw opening while producing vowels, and control for quantity, i.e., that translations are similar to the source in length. We perform an ablation study using an adversarial neural classifier which is trained to differentiate “true” dubbing translations from machine translations. While we are able to confirm the value of matching lip closure in dubbing, we find that the opening angle of the jaw as determined by the realized vowel may be less relevant than frequently considered in audio-visual translation. KW - ablation study KW - audiovisual translation KW - dubbing KW - lip synchrony KW - machine translation Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:gbv:18-228-7-2592 UR - http://ceur-ws.org/Vol-2989/short_paper46.pdf SP - 171 EP - 180 PB - CEUR Workshops CY - Amsterdam, the Netherlands ER -