TY - CHAP A1 - Hussein, Hussein A1 - Meyer-Sickendiek, Burkhard A1 - Baumann, Timo ED - Birkholz, Peter ED - Stone, Simon T1 - How to identify elliptical poems within a digital corpus of auditory poetry T2 - Elektronische Sprachsignalverarbeitung 2019 (ESSV), Tagungsband der 30. Konferenz, Dresden, 3.-8. März 2019 N2 - Ellipses denote the omission of one or more grammatically necessary phrases. In this paper, we will demonstrate how to identify such ellipses as a rhythmical pattern in modern and postmodern free verse poetry by using data from lyrikline which contain the corresponding audio recording of each poem as spoken by the original author. We present a feature engineering approach based on literary analysis as well as a neural networks based approach for the identification of ellipses within the lines of a poem. A contrast class to the ellipsis is defined from poems consisting of complete and correct sentences. The feature-based approach used features derived from a parser such as verb, comma, and sentence ending punctuation. The classifier of neural networks is trained on the line level to integrate the textual information, the spoken recitation, and the pause information between lines, and to integrate information across the lines within the poem. A statistic analysis of poet's gender showed that 65% of all elliptical poems were written by female poets. The best results, calculated by the weighted F-measure, for the classification of ellipsis with the contrast class is 0.94 with the neural networks based approach. The best results for classification of elliptical lines is 0.62 with the feature-based approach. Y1 - 2019 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:gbv:18-228-7-2579 UR - http://www.essv.de/paper.php?id=88 SN - 978-3-959081-57-3 SP - 247 EP - 254 PB - TUDpress CY - Dresden, Germany ER - TY - CHAP A1 - Baumann, Timo A1 - Meyer-Sickendiek, Burkhard A1 - Hussein, Hussein T1 - How to Identify Speech When Translating Unpunctuated Poetry T2 - Proceedings of Elektronische Sprachsignalverarbeitung: Tagungsband der 31. Konferenz, Magdeburg, 4.-6. März 2020 N2 - A large proportion of (post)-modern poetry contains no or hardly any punctuation. In our contribution, we will investigate how well punctuation information can be recovered for postmodern poetry based on the information contained in the text and speech of free verse poems. We use the world's largest corpus of spoken (post-)modern poetry from our partner lyrikline which contains the corresponding audio recording of each poem as spoken by the original author and features translations for many of the poems. We identify lines that contain a phrase break in the middle of the poetic line, which may already be helpful for philological analysis on one hand, and identify the position of the break in the line on the other hand. We select those poetic lines that contain one or more punctuation characters that typically indicate a phrase break in poetry (.,;:!?/) somewhere in the middle (rather than only at the end of the line) as our target class. We train a neural network (bidirectional recurrent neural network (RNN) based on gated recurrent units (GRU) with attention) that combines audio and textual features to identify the punctuation with the goal of applying it to reconstruct them within a corpus of unpunctuated poems. Our results clearly indicate that speech is helpful for recovering the constituency structure of post-modern poetry that is partially obfuscated by missing punctuation. Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:gbv:18-228-7-2587 UR - http://www.essv.de/paper.php?id=452 SP - 165 EP - 172 PB - Förderverein Elektronische Sprachsignalverabeitung e.V. CY - Magdeburg, Germany ER - TY - CHAP A1 - Baumann, Timo A1 - Saboo, Ashutosh ED - Ehrmann, Maud T1 - Evaluating Heuristics for Audio-Visual Translation T2 - Proceedings of the Conference of Computational Humanities Research, (CHR2021): November 17-19, 2021, Amsterdam, The Netherlands N2 - Dubbing, i.e., the lip-synchronous translation and revoicing of audio-visual media into a target language from a different source language, is essential for the full-fledged reception of foreign audio-visual media, be it movies, instructional videos or short social media clips. In this paper, we objectify influences on the ‘dubbability’ of translations, i.e., how well a translation would be synchronously revoiceable to the lips on screen. We explore the value of traditional heuristics used in evaluating the qualitative aspects, in particular matching bilabial consonants and the jaw opening while producing vowels, and control for quantity, i.e., that translations are similar to the source in length. We perform an ablation study using an adversarial neural classifier which is trained to differentiate “true” dubbing translations from machine translations. While we are able to confirm the value of matching lip closure in dubbing, we find that the opening angle of the jaw as determined by the realized vowel may be less relevant than frequently considered in audio-visual translation. KW - ablation study KW - audiovisual translation KW - dubbing KW - lip synchrony KW - machine translation Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:gbv:18-228-7-2592 UR - http://ceur-ws.org/Vol-2989/short_paper46.pdf SP - 171 EP - 180 PB - CEUR Workshops CY - Amsterdam, the Netherlands ER - TY - CHAP A1 - Baumann, Timo T1 - Coordinating Speech Delivery to Gesture Progress for Deictic Expressions with Incremental Speech Synthesis T2 - Proceedings of the Timing in Human-Robot Interaction Workshop held in conjunction with the 9th ACM/IEEE International Conference on Human-Robot Interaction (HRI 2014), Bielefeld, 3. March 2014 N2 - This paper presents ongoing work in incremental speech synthesis that enables a system to adapt speech delivery to unforeseen changes in the timing of motor events (e. g. a robot actuator working faster or slower than anticipated) in order to improve the coordination of speech and gestures for deictic expressions. KW - Incremental processing KW - speech synthesis KW - online adaptation KW - Inkrementelle Verarbeitung, KW - Sprachsynthese KW - Zeigegesten KW - Echtzeitadaptierung KW - Sprachverarbeitung Y1 - 2014 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:gbv:18-228-7-1991 UR - https://edoc.sub.uni-hamburg.de/informatik/volltexte/2014/199/pdf/timo_baumann_coordinating_speech_delivery.pdf ER -