TY - JOUR A1 - Freier, Carolin A1 - Bocklet, Tobias A1 - Helten, Anne-Kathrin A1 - Hoffmann, Franziska A1 - Hunger, Marianne A1 - Kovács, László A1 - Richter, Florian A1 - Riedhammer, Korbinian A1 - Schmohl, Tobias A1 - Simon, Claudia T1 - Wie kann videogestütztes Lernen die Erwartungen Studierender und Dozierender erfüllen? JF - Soziale Passagen N2 - ZusammenfassungIm BMBF-Verbundprojekt HAnS entwickeln und implementieren neun Hochschulen sowie drei hochschulübergreifende Einrichtungen ein intelligentes Hochschul-Assistenz-System als Open-Source-Lösung. Videobasierte Lehrmaterialien werden verschriftlicht und durch eine Indexierung Stichwortsuchen ermöglicht; geplant ist, über einen KI-Tutor automatisiert Übungsaufgaben zu generieren. Studierende sollen so in ihrem Selbststudium digital unterstützt werden. Die technische Entwicklung wird interdisziplinär – auch sozialwissenschaftlich und pädagogisch – begleitet und in einem iterativen Vorgehen evidenzbasiert entsprechend Design-Based-Research angepasst. Wissen und Wertesystem der Anwender*innen, Didaktik, Ethik, Akzeptanz und Datenschutz werden dabei im Entwicklungsprozess einbezogen. KW - Hochschulforschung · Intelligente Assistenzsysteme · Evaluation · Video-Lernen · Adressat*innen-zentrierte Technikentwicklung Y1 - 2023 U6 - https://doi.org/10.1007/s12592-023-00478-0 SN - 1867-0180 VL - 15 IS - 2 SP - 631 EP - 635 PB - Springer VS ER - TY - JOUR A1 - Schuster, Maria A1 - Arias-Vergara, Tomas A1 - Müller-Hörner, Rainer A1 - Winterholler, Cordula A1 - Bocklet, Tobias T1 - Verstehen mich mit der Maske noch alle? T1 - Speech quality changes due to face masks BT - Coronavirus-Pandemie JF - MMW - Fortschritte der Medizin N2 - In Arztpraxen nimmt der Infektionsschutz mittels Mund-Nasen-Maske momentan eine zentrale Rolle ein. Wenig nachgedacht wird dabei über die Folgen für die Kommunikation. Gerade schwerhörigen Patienten bereiten gedämpfte Stimmen und abgedeckte Lippen Probleme. KW - Face mask, speech quality, computer-assisted speech assessment, hearing loss Y1 - 2020 U6 - https://doi.org/10.1007/s15006-020-0749-4 VL - 162 IS - 14 SP - 42 EP - 44 ER - TY - JOUR A1 - Bocklet, Tobias A1 - Nöth, Elmar A1 - Riedhammer, Korbinian T1 - Künstliche Intelligenz für die Analyse pathologischer Sprache JF - Sprache - Stimme - Gehör N2 - Sprache kann eine Vielzahl von diagnostisch relevanten Informationen enthalten. In diesem Übersichtsartikel wird aufgezeigt, wie Methoden der Künstlichen Intelligenz, insbesondere Maschinelles Lernen und Sprachverarbeitung, angewendet auf Sprachsignale eingesetzt werden können: zur Bewertung von Verständlichkeit, zur Automatisierung von standardisierten Tests und zur Bestimmung medizinischer Skalen und Diagnosen. Eine abschließende kritischen Betrachtung von akustischen Merkmalen über eine Vielzahl von Pathologien gibt Grund zur Annahme, dass diese Marker tatsächlich diagnostisch relevante Informationen enthalten. N2 - Speech can contain a variety of diagnostically relevant cues. In this article, it is shown how artificial intelligence, in particular machine learning and speech processing, can be applied to speech signals: to assess intelligibility, to automate standardized tests, and to determine medical scales and diagnoses. We conclude with critical review of acoustic features across a variety of pathologies that give reason to believe that these markers do indeed contain diagnostically relevant information. T2 - Artificial Intelligence for the Analysis of Pathologic Speech KW - Künstliche Intelligenz - Machine Learning - pathologische Sprache - Spracherkennung - Sprachverarbeitung Y1 - 2023 U6 - https://doi.org/10.1055/a-2089-5778 VL - 47 IS - 3 ER - TY - CHAP A1 - Braun, Franziska A1 - Bayerl, Sebastian P. A1 - Pérez-Toro, Paula A. A1 - Hönig, Florian A1 - Lehfeld, Hartmut A1 - Hillemacher, Thomas A1 - Nöth, Elmar A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Classifying Dementia in the Presence of Depression BT - A Cross-Corpus Study N2 - Automated dementia screening enables early detection and intervention, reducing costs to healthcare systems and increasing quality of life for those affected. Depression has shared symptoms with dementia, adding complexity to diagnoses. The research focus so far has been on binary classification of dementia (DEM) and healthy controls (HC) using speech from picture description tests from a single dataset. In this work, we apply established baseline systems to discriminate cognitive impairment in speech from the semantic Verbal Fluency Test and the Boston Naming Test using text, audio and emotion embeddings in a 3-class classification problem (HC vs. MCI vs. DEM). We perform cross-corpus and mixed-corpus experiments on two independently recorded German datasets to investigate generalization to larger populations and different recording conditions. In a detailed error analysis, we look at depression as a secondary diagnosis to understand what our classifiers actually learn. Y1 - 2023 U6 - https://doi.org/10.48550/arXiv.2308.08306 ER - TY - CHAP A1 - Riedhammer, Korbinian A1 - Baumann, Ilja A1 - Bayerl, Sebastian P. A1 - Bocklet, Tobias A1 - Braun, Franziska A1 - Wagner’, Dominik T1 - Medical Speech Processing for Diagnosis and Monitoring BT - Clinical Use Cases N2 - In recent years, speech processing for medical applications got significant traction. While pioneering work in the 1990ies focused on processing sustained vowels or isolated utterances, work in the 2000s already showed, that speech recognition systems, prosodic analysis and natural language processing be used to assess a large variety of speech pathologies.Here, we give an overview of how to classify selected speech pathologies including stuttering, language development, speech intelligibility after surgery, dementia and Alzheimers, depression and state-of-mind. While each of those poses a rather well-defined problem in a lab setting, we discuss the issues when integrating such methods in a clinical workflow such as diagnosis or monitoring. Starting from the question if such detectors can be used for general screening or rather as a specialist's tool, we explore the legal and privacy-related implications: patient-doctor conversations, working with children or demented seniors, bias towards examiner or patient, on-device vs. cloud processing.We conclude with a set of open questions that should be addressed to help bringing all this research from the lab to routine clinical use. Y1 - 2023 SP - 1417 EP - 1420 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Braun, Franziska A1 - Bayerl, Sebastian P. A1 - Nöth, Elmar A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Multi-class Detection of Pathological Speech with Latent Features BT - How does it perform on unseen data? N2 - The detection of pathologies from speech features is usually defined as a binary classification task with one class representing a specific pathology and the other class representing healthy speech. In this work, we train neural networks, large margin classifiers, and tree boosting machines to distinguish between four pathologies: Parkinson's disease, laryngeal cancer, cleft lip and palate, and oral squamous cell carcinoma. We show that latent representations extracted at different layers of a pre-trained wav2vec 2.0 system can be effectively used to classify these types of pathological voices. We evaluate the robustness of our classifiers by adding room impulse responses to the test data and by applying them to unseen speech corpora. Our approach achieves unweighted average F1-Scores between 74.1% and 97.0%, depending on the model and the noise conditions used. The systems generalize and perform well on unseen data of healthy speakers sampled from a variety of different sources. Y1 - 2023 U6 - https://doi.org/10.21437/Interspeech.2023-464 SN - 2958-1796 SP - 2318 EP - 2322 ER - TY - CHAP A1 - Chen, Wenda A1 - Huang, Jonathan A1 - Bocklet, Tobias T1 - Length-and Noise-aware Training Techniques for Short-utterance Speaker Recognition N2 - Speaker recognition performance has been greatly improved with the emergence of deep learning. Deep neural networks show the capacity to effectively deal with impacts of noise and reverberation, making them attractive to far-field speaker recognition systems. The x-vector framework is a popular choice for generating speaker embeddings in recent literature due to its robust training mechanism and excellent performance in various test sets. In this paper, we start with early work on including invariant representation learning (IRL) to the loss function and modify the approach with centroid alignment (CA) and length variability cost (LVC) techniques to further improve robustness in noisy, far-field applications. This work mainly focuses on improvements for short-duration test utterances (1-8s). We also present improved results on long-duration tasks. In addition, this work discusses a novel self-attention mechanism. On the VOiCES far-field corpus, the combination of the proposed techniques achieves relative improvements of 7.0% for extremely short and 8.2% for full-duration test utterances on equal error rate (EER) over our baseline system. KW - speaker recognition, invariant representation learning, centroid alignment, x-vector, far-field Y1 - 2020 U6 - https://doi.org/10.48550/arXiv.2008.12218 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Bayerl, Sebastian P. A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Speaker Adaptation for End-To-End Speech Recognition Systems in Noisy Environments N2 - We analyze the impact of speaker adaptation in end-to-end automatic speech recognition models based on transformers and wav2vec 2.0 under different noise conditions. By including speaker embeddings obtained from x-vector and ECAPA-TDNN systems, as well as i-vectors, we achieve relative word error rate improvements of up to 16.3% on LibriSpeech and up to 14.5% on Switchboard. We show that the proven method of concatenating speaker vectors to the acoustic features and supplying them as auxiliary model inputs remains a viable option to increase the robustness of end-to-end architectures. The effect on transformer models is stronger, when more noise is added to the input speech. The most substantial benefits for systems based on wav2vec 2.0 are achieved under moderate or no noise conditions. Both x-vectors and ECAPA-TDNN embeddings outperform i-vectors as speaker representations. The optimal embedding size depends on the dataset and also varies with the noise condition. KW - speaker adaptation, automatic speech recognition, end-to-end systems, transformer, wav2vec 2.0 Y1 - 2023 U6 - https://doi.org/10.48550/arXiv.2211.08774 ER - TY - CHAP A1 - Wagner, Dominik A1 - Bayerl, Sebastian P. A1 - Bocklet, Tobias A1 - Draxler, Christoph T1 - Implementing Easy-to-Use Recipes for the Switchboard Benchmark N2 - We report on our contribution of templates for tokenization, language modeling, and automatic speech recognition (ASR) on the Switchboard benchmark to the open-source general-purpose toolkit SpeechBrain. Three recipes for the training of end-to-end ASR systems were implemented. We describe their model architectures, as well as the necessary data preparation steps. The word error rates achievable with our models are comparable to or better than those of other popular toolkits. Pre-trained ASR models were made available on HuggingFace. They can be easily integrated into research projects or used directly for quick inference via a hosted inference API. Y1 - 2023 SP - 150 EP - 157 PB - TUDpress, Dresden ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Riedhammer, Korbinian A1 - Nöth, Elmar A1 - Bocklet, Tobias T1 - Detection of Vowel Errors in Children’s Speech Using Synthetic Phonetic Transcripts T2 - 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Taipei, Taiwan, 2023 N2 - The analysis of phonological processes is crucial in evaluating speech development disorders in children, but encounters challenges due to limited children audio data. This work focuses on automatic vowel error detection using a two-stage pipeline. The first stage uses a fine-tuned cross-lingual phone recognizer (wav2vec 2.0) to extract phone sequences from audio. The second stage employs a language model (BERT) for classification from a phone sequence, entirely trained on synthetic transcripts, to counteract the very broad range of potential mistakes. We evaluate the system on nonword audio recordings recited by preschool children from a speech development test. The results show that the classifier trained on synthetic data performs well, but its efficacy relies on the quality of the phone recognizer. The best classifier achieves an 94.7% F1 score when evaluated against phonetic ground truths, whereas the F1 score is 76.2% when using automatically recognized phone sequences. KW - Conferences ; Pipelines ; Phonetics ; Audio recording ; Speech processing ; Synthetic data ; Automatic speech recognition ; children’s speech ; vowel errors ; nonwords Y1 - 2023 U6 - https://doi.org/10.1109/ASRU57964.2023.10389704 SP - 1 EP - 8 ER -