TY - JOUR A1 - Hammer, Simone A1 - Nunes, Danilo Weber A1 - Hammer, Michael A1 - Zeman, Florian A1 - Akers, Michael A1 - Götz, Andrea A1 - Balla, Annika A1 - Doppler, Michael Christian A1 - Fellner, Claudia A1 - Da Platz Batista Silva, Natascha A1 - Thurn, Sylvia A1 - Verloh, Niklas A1 - Stroszczynski, Christian A1 - Wohlgemuth, Walter Alexander A1 - Palm, Christoph A1 - Uller, Wibke T1 - Deep learning-based differentiation of peripheral high-flow and low-flow vascular malformations in T2-weighted short tau inversion recovery MRI JF - Clinical hemorheology and microcirculation N2 - BACKGROUND Differentiation of high-flow from low-flow vascular malformations (VMs) is crucial for therapeutic management of this orphan disease. OBJECTIVE A convolutional neural network (CNN) was evaluated for differentiation of peripheral vascular malformations (VMs) on T2-weighted short tau inversion recovery (STIR) MRI. METHODS 527 MRIs (386 low-flow and 141 high-flow VMs) were randomly divided into training, validation and test set for this single-center study. 1) Results of the CNN's diagnostic performance were compared with that of two expert and four junior radiologists. 2) The influence of CNN's prediction on the radiologists' performance and diagnostic certainty was evaluated. 3) Junior radiologists' performance after self-training was compared with that of the CNN. RESULTS Compared with the expert radiologists the CNN achieved similar accuracy (92% vs. 97%, p = 0.11), sensitivity (80% vs. 93%, p = 0.16) and specificity (97% vs. 100%, p = 0.50). In comparison to the junior radiologists, the CNN had a higher specificity and accuracy (97% vs. 80%, p <  0.001; 92% vs. 77%, p <  0.001). CNN assistance had no significant influence on their diagnostic performance and certainty. After self-training, the junior radiologists' specificity and accuracy improved and were comparable to that of the CNN. CONCLUSIONS Diagnostic performance of the CNN for differentiating high-flow from low-flow VM was comparable to that of expert radiologists. CNN did not significantly improve the simulated daily practice of junior radiologists, self-training was more effective. KW - magnetic resonance imaging KW - deep learning KW - Vascular malformation Y1 - 2024 U6 - https://doi.org/10.3233/CH-232071 SP - 1 EP - 15 PB - IOP Press ET - Pre-press ER - TY - CHAP A1 - Schuler, Christian A1 - Nayak, Shravan A1 - Saha, Debjoy A1 - Baumann, Timo ED - Baumann, Timo T1 - Can We See Your Response Before You Speak? Exploring Linguistic Information Found in Inter-Utterance Pauses T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - In this work we assess whether there is information in pauses in-between utterances of the same or different speakers that are predictive of the following speaker’s utterance. We present models that connect a person’s visual features before they speak to their upcoming utterance. In our experiments we find that outof-the-box pre-trained models can already reach a better-than-chance performance in correlating video embeddings to utterance embeddings. In contrast, models that attempt to predict the first word after the pause do not outperform a unigram model, indicating that our models do not read lips (based e.g. on co-articulation effects) but rather capture more fundamental aspects of the upcoming utterance. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-70949 SN - 978-3-95908-325-6 SP - 165 EP - 172 PB - TUDpress CY - Dresden ER - TY - CHAP A1 - Schubert, Martha A1 - Sinha, Yamini A1 - Krüger, Julia A1 - Siegert, Ingo ED - Baumann, Timo T1 - Speech Recognition Errors in ASR Engines and Their Impact on Linguistic Analysis in Psychotherapies T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - Modern intervention planning in psychotherapies may benefit from predicting process relevant psychotherapy constructs by automated speech analysis. One essential step is the extraction of relevant linguistic speech markers by ASR engines, which because of highly sensible data, work offline. We analyze transcription errors from NeMo, Whisper, and Wav2Vec2.0, focusing on their impact on linguistic markers usually requiring high quality transcripts. By utilizing part-of-speech tagging, we examine error occurrences among different word types. The Linguistic Inquiry and Word Count (LIWC) software aids in extracting markers. We highlight challenges in transcribing spontaneous speech, prevalent in therapy, and compare results with the Mozilla CommonVoice dataset, which features read speech. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-70999 SN - 978-3-95908-325-6 SP - 203 EP - 210 PB - TUDpress CY - Dresden ER - TY - CHAP A1 - Kettler, Benedict A1 - Hillmann, Stefan ED - Baumann, Timo T1 - Supervised vs. Zero-Shot Learning Automatic Classification of Comments on Educational Videos Using Pre-Trained Language Models T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - Despite the potential of AI, only a small percentage of small and medium-sized enterprises (SMEs) are adopting it due to data issues, expertise gaps, and implementation barriers. Zero-shot learning offers a promising approach for SMEs by minimizing these obstacles. This paper explores the use of zero-shot learning in a real-world NLP classification task on online comments (comparable with intent classification tasks) from the e-learning platform Sofatutor. While finetuning has achieved high accuracy (82.3–86.5%), zero-shot models have shown lower performance (39.3–61.4%) due to different label selection, grouping of different scenarios in one class and the type of classification task. Even if the current accuracy is not sufficient for practical application, pre-filtering the data using zeroshot learning might be a promising option for SMEs. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-70928 SN - 978-3-95908-325-6 SP - 149 EP - 156 PB - TUDpress CY - Dresden ER - TY - CHAP A1 - Venkateswaran, Siddarth A1 - Al Foysal, Abdullah A1 - Shaik, Nazeer Basha A1 - Böck, Ronald ED - Baumann, Timo T1 - Is there Text in Wine? – S+U Learning-based Named Entity Recognition and Triplet Extraction from Wine Aroma Descriptors T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - Wine making is usually considered a domain being far off the processing of speech and language. But in a particular aspect, the domains of speech processing and wine making are related, namely, in the description of wine aromas. These descriptors are used for creating wine expertise as well as more general (advertisement-like) textual representations. In the current paper, we use Natural Language Processing techniques, especially Named Entity Recognition, to identify Aspects and Opinions, reflecting wine characteristics. These are combined with analyses of respective relations (triplet extraction) building Aspect-Opinion-Pairs to establish indicative aroma descriptors, also trying to approach the complex interplay amongst these individual statements. In our experiments, we rely on the Falstaff corpus comprising a huge set of wine descriptions. This results in an average F1 score of around 0.85 for Aspect-Opinion classification. For triplet generation multiple strategies were compared, resulting in an average F1 score of 0.67 in this challenging task. For both tasks we rely only on a handful of manually annotated samples, applying pseudo-labeling methods from seed data to achieve automatic labeling. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-70931 SN - 978-3-95908-325-6 SP - 157 EP - 164 PB - TUDpress CY - Dresden ER - TY - CHAP A1 - Bauer, Judith A1 - Zalkow, Frank A1 - Müller, Meinard A1 - Dittmar, Christian ED - Baumann, Timo T1 - Evaluating the Impact of Prosody Feature Normalization on the Controllability of Pitch in Speech Synthesis T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - Recent neural text-to-speech (TTS) models are able to synthesize highly natural speech signals using deep learning techniques. In practical applications, it can be desirable to have explicit control over the prosody (speech rate, fundamental frequency, and energy) of the synthesized speech. Such controllability can be achieved by adding prosody prediction modules, whose main purpose is to estimate plausible prosody features for each phoneme in the text input. This explicit modeling also allows for changing prosody features at inference time, consequently enabling the adjustment of the prosody in the synthesized audio. In this paper, we evaluate to which extent deliberate manipulation of such prosody features is reflected in the resulting speech audio. We focus particularly on changing the pitch (i.e., fundamental frequency) while applying different normalization strategies. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-70976 SN - 978-3-95908-325-6 SP - 188 EP - 195 PB - TUDpress CY - Dresden ER - TY - CHAP A1 - Sinha, Yamini A1 - Hintz, Jan A1 - Siegert, Ingo ED - Baumann, Timo T1 - Evaluation of Audio Deepfakes – Systematic Review T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - Generative models for audio are commonly used for music composition, sound effects generation for video game development, audio restoration, voice cloning, etc. The ease of generating indistinguishable fake audio with deep learning poses a major threat to personal privacy, online security, and political discourse. Evaluating the quality and realism of these synthetic utterances is crucial for mitigating the potential for misinformation and harm. To assess this threat, this paper conducts a systematic review, using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), on how these deepfake models are currently evaluated. The analysis of 86 papers shows that the majority of the evaluation is conducted on a machine level and highlights a research gap regarding the human perception of deepfakes. This paper explores various methods and perceptual measures employed in assessing audio deepfakes and evaluating their strengths, limitations, and future directions. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-70960 SN - 978-3-95908-325-6 SP - 181 EP - 187 PB - TUDpress CY - Dresden ER - TY - CHAP A1 - Ibrahim, Omnia A1 - Yuen, Ivan A1 - Xue, Wei A1 - Andreeva, Bistra A1 - Möbius, Bernd ED - Baumann, Timo T1 - Listener-Oriented Consequences of Predictability-Based Acoustic Adjustment T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - This paper investigated whether predictability-based adjustments in production have listener-oriented consequences in perception. By manipulating the acoustic features of a target syllable in different predictability contexts in German, we tested 40 listeners’ perceptual preference for the manipulation. Four source words underwent acoustic modifications on the target syllable. Our results revealed a general preference for the original (unmodified) version over the modified one. However, listeners generally favored the unmodified version more when the source word had a higher predictable context compared to a less predictable one. The results showed that predictability-based adjustments have perceptual consequences and that listeners have predictability-based expectations in perception. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-70984 SN - 978-3-95908-325-6 SP - 196 EP - 202 PB - TUDpress CY - Dresden ER - TY - CHAP A1 - Mousavi, Neda A1 - Grawunder, Sven ED - Baumann, Timo T1 - The influence of signal segmentation methods on rhythm-bassed speaker recognition T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - This study investigates the effects of speech segmentation methods on speaker recognition models, particularly with regard to the use of rhythmic feature sets. Using three automatic methods and one manual method on the German database of Kiel corpus, segmentation was performed based on the identification of vowel onsets. Subsequently rhythmic variability indices derived from these intervals were calculated and used for principal component analysis and support vector machine model in order to investigate the variation among speakers. The results underline the influence of signal segmentation methods on speaker recognition models. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-71014 SN - 978-3-95908-325-6 SP - 225 EP - 232 PB - TUDpress CY - Dresden ER - TY - CHAP A1 - Venkateswaran, Siddarth A1 - Böck, Ronald ED - Baumann, Timo T1 - Can Language Models Behave Like Wine Sommeliers? – Using Multiple Agents To Evaluate The Quality of Wine Descriptors Generated By Llama 2 T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - Wines are complex beverages whose taste can be described either numerically or textually, with the former involving the rating of the intensities of different aroma characteristics often with the help of a wine tasting wheel, and the latter with the help of crisp terms often in a poetic fashion. These are often done with the help of wine sommeliers who with one sniff can describe the wine. Usually, each sommelier has a unique style when it comes to textually describing a wine, research has shown that such differences have no negative impact in correctly classifying wines on the basis of their color, grape variety, region etc. Given the recent advancements in the field of Natural Language Processing, especially with the emergence of Large Language Models, we aim to check the capability of Llama 2 in its ability to generate texts pertaining to a specific color of a wine, given a list of aroma intensities as input prompts. In our experiments, we relied on data from Meininger and Falstaff, and on a combination of domain adaptation and pseudo-labeling techniques to create the corpus to train the Llama 2 model on. Also, we relied on a voting scheme of three differently trained classifiers to evaluate the wine-color specific text generation capabilities of Llama 2. Additionally, we employed the services of domain experts to evaluate the quality of a sample set of texts that was generated by Llama 2. Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:898-opus4-70911 SN - 978-3-95908-325-6 SP - 141 EP - 148 PB - TUDpress CY - Dresden ER -