Externe Publikationen
Refine
Year of publication
- 2021 (5) (remove)
Document Type
- conference proceeding (article) (5) (remove)
Language
- English (5)
Has Fulltext
- no (5)
Is part of the Bibliography
- no (5)
Keywords
- Chinese older adults (1)
- Distributed intelligent home (1)
- Gerontechnology (1)
- Population aging (1)
- Smart furniture (1)
- Survey (1)
- Untertitelung (1)
- VoIP (1)
- ablation study (1)
- audiovisual translation (1)
Institute
Begutachtungsstatus
- peer-reviewed (5)
Dubbing, i.e., the lip-synchronous translation and revoicing of audio-visual media into a target language from a different source language, is essential for the full-fledged reception of foreign audio-visual media, be it movies, instructional videos or short social media clips. In this paper, we objectify influences on the ‘dubbability’ of translations, i.e., how well a translation would be synchronously revoiceable to the lips on screen. We explore the value of traditional heuristics used in evaluating the qualitative aspects, in particular matching bilabial consonants and the jaw opening while producing vowels, and control for quantity, i.e., that translations are similar to the source in length. We perform an ablation study using an adversarial neural classifier which is trained to differentiate “true” dubbing translations from machine translations. While we are able to confirm the value of matching lip closure in dubbing, we find that the opening angle of the jaw as determined by the realized vowel may be less relevant than frequently considered in audio-visual translation.
We present an open source plugin for live subtitling in the popular open source video conferencing software BigBlueBut-ton. Our plugin decodes each speaker’s audio stream separately and in parallel, thereby obliviating the need for speaker di-arization and seamlessly handling overlapped talk. Any Kaldi-compatible nnet3 model can be used with our plugin and we demonstrate it using freely available TDNN-HMM-based ASR models for English and German. Our subtitles can be used as they are (e.g., in loud environments) or can form the basis for further NLP processes. Our tool can also simplify the collection of remotely recorded multi-party dialogue corpora.
In this paper, we present a study in which a robot initiates interactions with people passing by in an in-the-wild scenario. The robot adapts the loudness of its voice dynamically to the distance of the respective person approached, thus indicating who it is talking to. It furthermore tracks people based on information on body orientation and eye gaze and adapts the text produced based on people's distance autonomously. Our study shows that the adaptation of the loudness of its voice is perceived as personalization by the participants and that the likelihood that they stop by and interact with the robot increases when the robot incrementally adjusts its behavior.
Population aging is one of the major challenges facing the world. In particular, the advent of China's aging society caused by various factors will be a major threat to its future development. Therefore, serious measures need to be taken to achieve its demographic sustainability. Smart furniture can be considered as a novel subcategory of gerontechnology. One of the main outcomes of the EU-funded REACH project was a variety of smart furniture named Personalized Intelligent Interior Units (PI2Us) which served as the key component of a distributed intelligent home to promote the health and activity level of older adults. This outcome can potentially be a solution to mitigate the consequences caused by population aging. In order to understand the attitudes and opinions of Chinese older adults towards the relevant technologies, the authors conducted an opinion survey using the PI2Us as an example, which sampled more than 380 older adults in 26 out of 34 provincial-level administrative divisions of China. The survey showed that Chinese older adults in general have a highly positive attitude towards smart furniture and smart home technologies. Several other insights also can be revealed from the survey. Based on further analyses, the paper summarized why the elderly-oriented smart furniture and distributed intelligent home has the potential to thrive in China's market soon. Finally, a three-year project action plan for implementing localized solutions in cooperation with a large Chinese furniture manufacturer was presented.
We present a fully automatic solution for German video subtitling, with a focus on lecture videos. We rely entirely on open source models and scripts for German ASR, automatic punctuation reconstruction and subtitle segmentation. All training scripts, 1000h of German speech training data, pre-trained models and the final subtitling program are publicly available. It can readily be integrated into lecture video platforms such as Lecture2Go. The automatically generated subtitles can also serve as a basis to make the video material more accessible (e.g. via search, keyword clouds, and the like) or for further manual revision, potentially helping in significantly speeding up manual work. A particular challenge that we observe in lectures are technical terms that are frequent in a particular lecture, but infrequent in a typical language model and that might be out of vocabulary for a general purpose ASR. We approach this challenge by extracting texts from accompanying lecture slides to adapt the language model of our TDNN-HMM based ASR system. We demonstrate the usability of the full system and its generated subtitles and evaluate on a dataset of manually transcribed lectures with an average of 26.3% WER.