TY - CHAP A1 - Baumann, Ilja A1 - Unger, Nicole A1 - Wagner, Dominik A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Automatic Evaluation of a Sentence Memory Test for Preschool Children N2 - Assessment of memory capabilities in preschool-aged children is crucial for early detection of potential speech development impairments or delays. We present an approach for the automatic evaluation of a standardized sentence memory test specifically for preschool children. Our methodology leverages automatic transcription of recited sentences and evaluation based on natural language processing techniques. We demonstrate the effectiveness of our approach on a dataset comprised of recited sentences from preschool-aged children, incorporating ratings of semantic and syntactic correctness. The best performing systems achieve an F1 score of 91.7% for semantic correctness and 86.1% for syntactic correctness using automatic transcripts. Our results showcase the potential of automated evaluation systems in providing reliable and efficient assessments of memory capabilities in early childhood, facilitating timely interventions and support for children with language development needs. KW - speech development, children’s speech, automatic assessment Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2125 SP - 5158 EP - 5162 ER - TY - JOUR A1 - Schmitt, Thomas A1 - Bundscherer, Maximilian A1 - Bocklet, Tobias T1 - Training a Computer Vision Model for Commercial Bakeries with Primarily Synthetic Images N2 - In the food industry, reprocessing returned product is a vital step to increase resource efficiency. [SBB23] presented an AI application that automates the tracking of returned bread buns. We extend their work by creating an expanded dataset comprising 2432 images and a wider range of baked goods. To increase model robustness, we use generative models pix2pix and CycleGAN to create synthetic images. We train state-of-the-art object detection model YOLOv9 and YOLOv8 on our detection task. Our overall best-performing model achieved an average precision AP@0.5 of 90.3% on our test set. KW - machine learning, object detection, YOLOv9, image composition, baked goods, food industry, industrial automation Y1 - 2024 U6 - https://doi.org/10.48550/arXiv.2409.20122 ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Schuster, Maria A1 - Riedhammer, Korbinian A1 - Nöth, Elmar A1 - Bocklet, Tobias T1 - Towards Self-Attention Understanding for Automatic Articulatory Processes Analysis in Cleft Lip and Palate Speech N2 - Cleft lip and palate (CLP) speech presents unique challenges for automatic phoneme analysis due to its distinct acoustic characteristics and articulatory anomalies. We perform phoneme analysis in CLP speech using a pre-trained wav2vec 2.0 model with a multi-head self-attention classification module to capture long-range dependencies within the speech signal, thereby enabling better contextual understanding of phoneme sequences. We demonstrate the effectiveness of our approach in the classification of various articulatory processes in CLP speech. Furthermore, we investigate the interpretability of self-attention to gain insights into the model’s understanding of CLP speech characteristics. Our findings highlight the potential of the selfattention mechanisms for improving automatic phoneme analysis in CLP speech, paving the way for enhanced diagnostics, adding interpretability for therapists and affected patients. KW - pathologic speech, cleft lip and palate, children’s speech, automatic assessment Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2134 SP - 2430 EP - 2434 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias ED - Pesak, Krisztina T1 - Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models N2 - This paper explores the improvement of post-training quantization (PTQ) after knowledge distillation in the Whisper speech foundation model family. We address the challenge of outliers in weights and activation tensors, known to impede quantization quality in transformer-based language and vision models. Extending this observation to Whisper, we demonstrate that these outliers are also present when transformer-based models are trained to perform automatic speech recognition, necessitating mitigation strategies for PTQ. We show that outliers can be reduced by a recently proposed gating mechanism in the attention blocks of the student model, enabling effective 8-bit quantization, and lower word error rates compared to student models without the gating mechanism in place. KW - post-training quantization, Whisper, gated attention, outliers Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2105 SP - 4623 EP - 4627 ER - TY - JOUR A1 - Freier, Carolin A1 - Bocklet, Tobias A1 - Helten, Anne-Kathrin A1 - Hoffmann, Franziska A1 - Hunger, Marianne A1 - Kovács, László A1 - Richter, Florian A1 - Riedhammer, Korbinian A1 - Schmohl, Tobias A1 - Simon, Claudia T1 - Wie kann videogestütztes Lernen die Erwartungen Studierender und Dozierender erfüllen? JF - Soziale Passagen N2 - ZusammenfassungIm BMBF-Verbundprojekt HAnS entwickeln und implementieren neun Hochschulen sowie drei hochschulübergreifende Einrichtungen ein intelligentes Hochschul-Assistenz-System als Open-Source-Lösung. Videobasierte Lehrmaterialien werden verschriftlicht und durch eine Indexierung Stichwortsuchen ermöglicht; geplant ist, über einen KI-Tutor automatisiert Übungsaufgaben zu generieren. Studierende sollen so in ihrem Selbststudium digital unterstützt werden. Die technische Entwicklung wird interdisziplinär – auch sozialwissenschaftlich und pädagogisch – begleitet und in einem iterativen Vorgehen evidenzbasiert entsprechend Design-Based-Research angepasst. Wissen und Wertesystem der Anwender*innen, Didaktik, Ethik, Akzeptanz und Datenschutz werden dabei im Entwicklungsprozess einbezogen. KW - Hochschulforschung · Intelligente Assistenzsysteme · Evaluation · Video-Lernen · Adressat*innen-zentrierte Technikentwicklung Y1 - 2023 U6 - https://doi.org/10.1007/s12592-023-00478-0 SN - 1867-0180 VL - 15 IS - 2 SP - 631 EP - 635 PB - Springer VS ER - TY - CHAP A1 - Ranzenberger, Thomas A1 - Freier, Carolin A1 - Reinold, Luca A1 - Riedhammer, Korbinian A1 - Schneider, Fabian A1 - Simic, Christopher A1 - Simon, Claudia A1 - Freisinger, Steffen A1 - Georges, Munir A1 - Bocklet, Tobias T1 - A Multidisciplinary Approach to AI-based self-motivated Learning and Teaching with Large Language Models T2 - Proceedings of DELFI 2024 N2 - We present a learning experience platform that uses machine learning methods to support students and lecturers in self-motivated online learning and teaching processes. The platform is being developed as an agile open-source collaborative project supported by multiple universities and partners. The development is guided didactically, reviewed, and scientifically evaluated in several cycles. Transparency, data protection and the copyright compliant use of the system is a central part of the project. The system further employs large language models (LLMs). Due to privacy concerns, we utilize locally hosted LLM instances and explicitly do not rely on available cloud products. Students and lecturers can interact with an LLM-based chatbot in the current prototype. The AI-generated outputs contain cross-references to the current educational video’s context, indicating if sections are based on the lectures context or world knowledge. We present the prototype and results of our qualitative evaluation from the perspective of lecturers and students. KW - Artificial Intelligence in Education; Learning Experience Platform; Open Source Software; Large Language Models Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024_11 SN - 2944-7682 PB - Gesellschaft für Informatik e.V. CY - Bonn ER - TY - JOUR A1 - Lippert, Carolyn A1 - Rudolph, Eric A1 - Poltermann, Aleksandra A1 - Engert, Natalie A1 - Lehmann, Robert A1 - Albrecht, Jens T1 - Generative KI in der beraterischen Ausbildung BT - Der Einsatz eines*r virtuellen Klient*in als Übungstool für angehende Onlineberater*innen JF - e-beratungsjournal.net – Zeitschrift für Online-Beratung und computervermittelte Kommunikation N2 - Der Beitrag beleuchtet das Projekt "Virtuelle*r Klient*in (ViKl)" am Institut für E-Beratung der Technischen Hochschule Nürnberg Georg Simon Ohm, das sich mit der Nutzung eines KI-basierten Chatbots für die Simulation von Klient*innen in der Lehrveranstaltung "Querschnitt Onlineberatung" im Bachelor-Studiengang Soziale Arbeit befasst. Ziel ist es, praxisnahe Erfahrungen zu vermitteln und die Fähigkeiten der Studierenden im digitalen Beratungsgespräch zu verbessern. Durch den Einsatz verschiedener maschineller Lernarchitekturen werden passende Antworten generiert, die zum Gesprächsverlauf und zur psychosozialen Problemstellung passen. Die Nutzung einer umfangreichen Datenbasis aus dem Zertifikatskurs "Onlineberatung" sowie ein zielgruppenadäquates didaktisches Konzept tragen zur Qualität des*der ViKl bei. Die Diskussion umfasst auch zukünftige Entwicklungen in der Onlineberatung, die didaktische Gestaltung von KI-basierten Lehr-Lern-Formaten, sowie Herausforderungen und Vorteile dieser Ansätze. KW - Generative KI KW - Hochschule KW - Lehre KW - Computerunterstützte Kommunikation KW - Psychosoziale Beratung KW - Onlinekommunikation KW - digitale Kommunikation KW - Generative KI KW - Onlineberatung KW - Chatberatung Y1 - 2024 U6 - https://doi.org/10.48341/tcgc-st69 SN - 1816-7632 VL - 20 IS - 1 ER - TY - JOUR A1 - Escobar-Grisales, Daniel A1 - Ríos-Urrego, Cristian-David A1 - Baumann, Ilja A1 - Riedhammer, Korbinian A1 - Nöth, Elmar A1 - Bocklet, Tobias A1 - Garcia, Adolfo A1 - Orozco-Arroyave, Juan rafael T1 - It’s Time to Take Action: Acoustic Modeling of Motor Verbs to Detect Parkinson’s Disease N2 - Pre-trained models generate speech representations that are used in different tasks, including the automatic detection of Parkinson’s disease (PD). Although these models can yield high accuracy, their interpretation is still challenging. This paper used a pre-trained Wav2vec 2.0 model to represent speech frames of 25ms length and perform a frame-by-frame discrimination between PD patients and healthy control (HC) subjects. This fine granularity prediction enabled us to identify specific linguistic segments with high discrimination capability. Speech representations of all produced verbs were compared w.r.t. nouns and the first ones yielded higher accuracies. To gaina deeper understanding of this pattern, representations of motor and non-motor verbs were compared and the first ones yielded better results, with accuracies of around 83% in an independent test set. These findings support well-established neurocognitive models about action-related language highlighted as key drivers of PD. Index Terms: computational paralinguistics, interpretability of pre-trained models, action verbs, Parkinson’s disease KW - computational paralinguistics, interpretability of pre-trained models, action verbs, Parkinson’s disease Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2205 ER - TY - CHAP A1 - Bayerl, Sebastian Peter A1 - Wagner, Dominik A1 - Noeth, Elmar A1 - Riedhammer, Korbinian T1 - Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0 T2 - Interspeech 2022 N2 - Stuttering is a varied speech disorder that harms an individual’s communication ability. Persons who stutter (PWS) often use speech therapy to cope with their condition. Improving speech recognition systems for people with such non-typical speech or tracking the effectiveness of speech therapy would require systems that can detect dysfluencies while at the same time being able to detect speech techniques acquired in therapy. This paper shows that fine-tuning wav2vec 2.0 [1] for the classification of stuttering on a sizeable English corpus containing stuttered speech, in conjunction with multi-task learning, boosts the effectiveness of the general-purpose wav2vec 2.0 features for detecting stuttering in speech; both within and across languages. We evaluate our method on FluencyBank , [2] and the German therapy-centric Kassel State of Fluency (KSoF) [3] dataset by training Support Vector Machine classifiers using features extracted from the finetuned models for six different stuttering-related event types: blocks, prolongations, sound repetitions, word repetitions, interjections, and – specific to therapy – speech modifications. Using embeddings from the fine-tuned models leads to relative classification performance gains up to 27% w.r.t. F1-score. KW - stuttering KW - pathological speech KW - computational paralinguistics Y1 - 2022 U6 - https://doi.org/10.21437/Interspeech.2022-10908 SP - 2868 EP - 2872 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Bayerl, Sebastian Peter A1 - Roccabruna, Gabriel A1 - Chowdhury, Shammur Absar A1 - Ciulli, Tommaso A1 - Danieli, Morena A1 - Riedhammer, Korbinian A1 - Riccardi, Giuseppe T1 - What can Speech and Language Tell us About the Working Alliance in Psychotherapy T2 - Interspeech 2022 N2 - We are interested in the problem of conversational analysis and its application to the health domain. Cognitive Behavioral Therapy is a structured approach in psychotherapy, allowing the therapist to help the patient to identify and modify the malicious thoughts, behavior, or actions. This cooperative effort can be evaluated using the Working Alliance Inventory Observer-rated Shortened – a 12 items inventory covering task, goal, and relationship – which has a relevant influence on therapeutic outcomes. In this work, we investigate the relation between this alliance inventory and the spoken conversations (sessions) between the patient and the psychotherapist. We have delivered eight weeks of e-therapy, collected their audio and video call sessions, and manually transcribed them. The spoken conversations have been annotated and evaluated with WAI ratings by professional therapists. We have investigated speech and language features and their association with WAI items. The feature types include turn dynamics, lexical entrainment, and conversational descriptors extracted from the speech and language signals. Our findings provide strong evidence that a subset of these features are strong indicators of working alliance. To the best of our knowledge, this is the first and a novel study to exploit speech and language for characterising working alliance. KW - conversational analysis KW - working alliance KW - psychotherapy Y1 - 2022 U6 - https://doi.org/10.21437/Interspeech.2022-347 SP - 2443 EP - 2447 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Burghardt, Jennifer A1 - Stieler, Mara A1 - Steigerwald, Philipp A1 - Rudolph, Eric A1 - Albrecht, Jens A1 - Lehmann, Robert T1 - KI-gestützte Beratung – Assistenzsysteme für Ratsuchende und Fachkräfte in der digitalen Beratung T2 - Deutscher Suchtkongress (DSK) Y1 - 2025 U6 - https://doi.org/10.18416/DSK.2025.2259 VL - 2 IS - 1 ER - TY - BOOK A1 - Axenie, Cristian A1 - Bauer, Roman A1 - López Corona, Oliver A1 - West, Jeffrey T1 - Applied Antifragility in Natural Systems BT - From Principles to Applications N2 - As coined in the book of Nassim Taleb, antifragility is a property of a system to gain from uncertainty, randomness, and volatility, opposite to what fragility would incur. An antifragile system’s response to external perturbations is beyond robust, such that small stressors can strengthen the future response of the system by adding a strong anticipation component. Such principles are already well suited for describing behaviors in natural systems but also in approaching therapy designs and eco-system modelling and eco-system analysis. The purpose of this book is to build a foundational knowledge base by applying antifragile system design, analysis, and development in natural systems, including biomedicine, neuroscience, and ecology as main fields. We are interested in formalizing principles and an apparatus that turns the basic concept of antifragility into a tool for designing and building closed-loop systems that behave beyond robust in the face of uncertainty when characterizing and intervening in biomedical and ecological (eco)systems. The book introduces the framework of applied antifragility and possible paths to build systems that gain from uncertainty. We draw from the body of literature on natural systems (e.g. cancer therapy, antibiotics, neuroscience, and agricultural pest management) in an attempt to unify the scales of antifragility in one framework. The work of the Applied Antifragility Group in oncology, neuroscience, and ecology led by the authors provides a good overview on the current research status. T2 - with a foreword from Nassim Taleb Y1 - 2025 SN - 9783031903908 U6 - https://doi.org/10.1007/978-3-031-90391-5 SN - 2191-5768 PB - Springer Nature Switzerland CY - Cham ER - TY - BOOK A1 - Axenie, Cristian A1 - Akbarzadeh, Meisam A1 - Makridis, Michail A. A1 - Saveriano, Matteo A1 - Stancu, Alexandru T1 - Applied Antifragility in Technical Systems BT - From Principles to Applications N2 - The book purpose is to build a foundational knowledge base by applying antifragile system design, analysis, and development in technical systems, with a focus on traffic engineering, robotics, and control engineering. The authors are interested in formalizing principles and an apparatus that turns the basic concept of antifragility into a tool for designing and building closed-loop technical systems that behave beyond robust in the face of uncertainty. As coined in the book of Nassim Taleb, antifragility is a property of a system to gain from uncertainty, randomness, and volatility, opposite to what fragility would incur. An antifragile system’s response to external perturbations is beyond robust, such that small stressors can strengthen the future response of the system by adding a strong anticipation component. The work of the Applied Antifragility Group in traffic control and robotics, led by the authors, provides a good overview on the current research status. T2 - with a foreword from Nassim Taleb Y1 - 2025 SN - 9783031904240 U6 - https://doi.org/10.1007/978-3-031-90425-7 SN - 2191-5768 PB - Springer Nature Switzerland CY - Cham ER -