TY - RPRT A1 - Albrecht, Jens A1 - Lehmann, Robert A1 - Zauter, Sigrid T1 - Digitale Assistenz in der psychosozialen Beratung BT - Projekt-Abschlussbericht ; 9.2019-2.2021 N2 - In vielen Unternehmen werden heutzutage Chatbots eingesetzt, um Standard-Anfragen von Kunden effizient zu beantworten. Auch im psychosozialen Beratungsumfeld können natürlichsprachliche digitale Assistenzen eine sinnvolle Ergänzung zu menschlichen Beratern sein, denn sie sind jederzeit verfügbare, anonyme Ansprechpartner. Die Hürden für den Einsatz in der psychosozialen Beratung sind allerdings hoch, da die Entwicklung und der Betrieb von Chatbots zunächst Kosten verursacht und systematische Studien zum Mehrwehrt derartiger Systeme in diesem Anwendungsfeld bisher nicht vorliegen. Ziel des Projekts DIA (Digitale Assistenz) war die prototypische Entwicklung eines lernfähigen Chatbots für die psychosoziale Beratung. Dabei wurde insbesondere die Frage untersucht, wie aufwändig es ist, einen KI-basierten Chatbot ausschließlich mit Open-Source-Technologie zu entwickeln. Das von der Staedtler-Stiftung geförderte Projekt lief von September 2019 bis Februar 2021. Die Arbeiten wurden in einer interdisziplinären Kooperation vom Institut für E-Beratung mit der Fakultät Informatik der TH Nürnberg durchgeführt. Als Anwendungsdomäne diente die Bildungsberatung für den Zertifikatskurs Online-Beratung, die bisher hauptsächlich über E-Mail-Kommunikation erfolgt. T3 - E-Beratung - 2021,1 KW - Onlineberatung, Onlinekommunikation, digitale Kommunikation, Chatbot KW - Psychosoziale Beratung KW - computerunterstützte Kommunikation KW - ChatBot Y1 - 2021 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:92-opus4-7792 VL - 2021 IS - 1 ER - TY - RPRT A1 - Lehmann, Robert A1 - Albrecht, Jens A1 - Zauter, Sigrid T1 - CaSoTex, Computerunterstützte Analyse Sozialwissenschaftlicher Texte T2 - E-Beratung N2 - Ziel des Forschungsprojektes war die Erforschung der Einsatzmöglichkeiten computerlinguistischer Methoden und maschineller Lernverfahren zur automatisierten Analyse sozialwissenschaftlicher Texte. Die Datengrundlage bildeten digital vorliegende Beiträge aus moderierten, sozialen Beratungsforen. Anhand konkreter sozial-wissenschaftlicher Fragestellungen zu Wirkungszusammenhängen in der Onlineberatung sollte untersucht werden, inwiefern Computer-Linguistik und maschinelle Lernverfahren qualitative Analysen unterstützen bzw. ergänzen können, wo die Grenzen der Verfahren liegen und wie bei deren Einsatz vorzugehen ist. Im Projekt wurden erste konkrete Erkenntnisse generiert, der Forschungsverbund vergrößerte sich, Praxis-Forschungsprojekte wurden beantragt und es entstanden weitere Überlegungen zu größeren Forschungsprojekten. Die aktuellen Ergebnisse sind vielversprechend. Überwachte Lernverfahren können mit ausreichend vielen und gut vorbereiteten Trainingsdaten interpretierbare und verwertbare Erkenntnisse liefern. T3 - E-Beratung - 2020,1 KW - Künstliche Intelligenz KW - Computerunterstützte Kommunikation KW - Onlineberatung KW - Onlinekommunikation KW - digitale Kommunikation Y1 - 2020 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:92-opus4-6197 VL - 2020 IS - 1 CY - Nürnberg ER - TY - CHAP A1 - Braun, Franziska A1 - Erzigkeit, Andreas A1 - Lehfeld, Hartmut A1 - Hillemacher, Thomas A1 - Riedhammer, Korbinian A1 - Bayerl, Sebastian P. T1 - Going Beyond the Cookie Theft Picture Test: Detecting Cognitive Impairments Using Acoustic Features T2 - Text, Speech, and Dialogue Y1 - 2022 SN - 9783031162695 SN - 978-3-031-16270-1 U6 - https://doi.org/10.1007/978-3-031-16270-1_36 SN - 0302-9743 SP - 437 EP - 448 PB - Springer International Publishing CY - Cham ER - TY - CHAP A1 - Bayerl, Sebastian P. A1 - Frassetto, Tommaso A1 - Jauernig, Patrick A1 - Riedhammer, Korbinian A1 - Sadeghi, Ahmad-Reza A1 - Schneider, Thomas A1 - Stapf, Emmanuel A1 - Weinert, Christian T1 - Offline Model Guard: Secure and Private ML on Mobile Devices T2 - 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE) N2 - Performing machine learning tasks in mobile applications yields a challenging conflict of interest: highly sensitive client information (e.g., speech data) should remain private while also the intellectual property of service providers (e.g., model parameters) must be protected. Cryptographic techniques offer secure solutions for this, but have an unacceptable overhead and moreover require frequent network interaction.In this work, we design a practically efficient hardware-based solution. Specifically, we build OFFLINE MODEL GUARD (OMG) to enable privacy-preserving machine learning on the predominant mobile computing platform ARM—even in offline scenarios. By leveraging a trusted execution environment for strict hardware-enforced isolation from other system components, OMG guarantees privacy of client data, secrecy of provided models, and integrity of processing algorithms. Our prototype implementation on an ARM HiKey 960 development board performs privacy-preserving keyword recognition using TensorFlow Lite for Microcontrollers in real time. Y1 - 2020 U6 - https://doi.org/10.23919/DATE48585.2020.9116560 PB - IEEE ER - TY - CHAP A1 - Rudolph, Eric A1 - Seer, Hanna A1 - Mothes, Carina A1 - Albrecht, Jens T1 - Automated feedback generation in an intelligent tutoring system for counselor education T2 - Annals of Computer Science and Information Systems N2 - This paper investigates the implementation of AI- driven feedback in an intelligent tutoring system (ITS) developed for training of counselors. By using LLMs, the study explores the automatic generation of feedback for communication-intensive tasks such as online counseling. The evaluation compares dif- ferent feedback methods, including the sandwich, WWW and STATE methods, and assesses their emotional and objective impact. The results show that AI-generated feedback fulfills objective criteria better than emotional ones. Fine-tuning an open source LLM can improve both the emotional and objective quality of feedback. Furthermore, the study examines the accep- tance of AI feedback among aspiring counselors, highlighting the influence of familiarity with AI on acceptance levels. Ethical con- siderations, including bias and hallucination, are addressed, with recommendations for risk mitigation through multi-feedback options and expert supervision. This research contributes to the understanding of the role of AI in improving digital counseling practices and highlights the need for continuous evaluation and ethical considerations. KW - Intelligent Tutoring System KW - Automated Feedback Generation KW - Artificial Intelligence KW - Counselling Y1 - 2024 U6 - https://doi.org/10.15439/2024F1649 SN - 2300-5963 VL - 39 SP - 501 EP - 512 PB - IEEE ER - TY - CHAP A1 - Albrecht, Jens A1 - Rudolph, Eric A1 - Poltermann, Aleksandra A1 - Lehmann, Robert T1 - The Virtual Client: Leveraging Generative AI for Innovative Online-Counselor Education T2 - ICERI Proceedings N2 - In an increasingly intricate world, individuals turn to online platforms for guidance from trained counselors regarding various psychological and social challenges. The training of competent and effective counselors is challenging, requiring specific strategies, methods and realistic role-playing scenarios. VirCo (Virtual Client for Online Counseling) is an innovative educational platform that employs an open-source large language model (LLM) to simulate clients facing psycho-social problems. VirCo enables learners to engage with a variety of client personas through realistic conversations, thereby enhancing their counseling abilities in a self-directed manner. Furthermore, it features tools for course development, assignments, and multiple feedback systems, fostering a collaborative learning environment. This innovative use of AI technology offers scalable and practical training solutions, contributing to a skilled and proficient workforce in online counseling. KW - Education, Role-Play, Counseling, Generative AI Y1 - 2024 U6 - https://doi.org/10.21125/iceri.2024.0454 SN - 2340-1095 SP - 1522 EP - 1531 PB - IATED ER - TY - CHAP A1 - Lehmann, Robert A1 - Albrecht, Jens A1 - Zauter, Sigrid T1 - Die Computerunterstützte Analyse Sozialwissenschaftlicher Texte – Ergebnisse des Forschungsprojekts „Casotex“ T2 - Gegenwart und Zukunft sozialer Dienstleistungsarbeit N2 - Ein Datensatz aus mehreren deutschsprachigen Onlineberatungsforen wurde mit verschiedenen maschinellen Lernverfahren untersucht. Dabei können unüberwachte Lernverfahren nur in Bezug auf die linguistische Gestaltung der Beiträge relevante Ergebnisse zeigen. Überwachte maschinelle Lernverfahren zeigen ein sehr großes Potenzial zur Klassifikation von Textstellen analog zur qualitativen Inhaltsanalyse. Damit eignen sich maschinelle Lernverfahren prinzipiell für die qualitative Analyse großer Textmengen, wie sie z. B. in der Onlineberatung vorliegen. Y1 - 2021 SN - 9783658325558 U6 - https://doi.org/10.1007/978-3-658-32556-5_12 SN - 2569-2127 SP - 167 EP - 180 PB - Springer Fachmedien Wiesbaden CY - Wiesbaden ER - TY - RPRT A1 - Albrecht, Jens A1 - Lehmann, Robert A1 - Poltermann, Aleksandra T1 - GeCCo 1.0 - Erstellung eines öffentlichen Datensatzes für die KI-basierte Inhaltsanalyse in der Online-Beratung T2 - Schriftenreihe der Technischen Hochschule Nürnberg Georg Simon Ohm N2 - Ein Einsatzfeld Künstlicher Intelligenz im Bereich der Sozialen Arbeit ist die automatische Inhaltsanalyse von Konversationen aus der psychosozialen Onlineberatung. Mögliche Anwendungen sind die statistische Untersuchung von Wirkzusammenhängen sowie die Entwicklung KI-basierter Dialog-und Unterstützungssysteme. Die Grundlage für das Training entsprechender Modelle sind aussagekräftige und realistische Trainingsdaten. Im Rahmen des Projektes GeCCo (German e-Counseling Conversation Dataset) wurde ein erster deutschsprachiger Datensatz für die Analyse psychosozialer Beratungskonversationen erstellt und veröffentlicht. Wesentlicher Bestandteil ist ein neu entwickeltes Kategoriensystem, das eine feingranulare inhaltliche Klassifikation in 40 Berater-und 28 Klienten-Kategorien ermöglicht. Basierend auf dem Datensatz konnten erfolgreich verschiedene Modelle mithilfe maschineller Lernverfahren trainiert und für die wissenschaftliche Nutzung bereitgestellt werden. KW - Online-Beratung KW - KI KW - Datensatz KW - automatische Inhaltsanalyse Y1 - 2024 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:92-opus4-13894 SP - 3 EP - 15 ER - TY - JOUR A1 - Axenie, Cristian A1 - Halilov, Ertan A1 - Main, Julian A1 - Weiss, David T1 - Edge neuro-statistical learning for event-based visual motion detection and tracking in roadside safety systems JF - Neuromorphic Computing and Engineering N2 - The Vision Zero Program's purpose is to reduce traffic-related fatalities and serious injuries while promoting equitable, safe, and healthy mobility for all. Ultimately, the challenge is to detect pedestrians during the day and especially at night to implement safety measures. The current study introduces an award-winning low-power solution employing neuromorphic visual sensing and hybrid neuro-statistical processing developed by the Technische Hochschule Nürnberg team for the TinyML Vision Zero San Jose Competition. The solution proposes a novel neuromorphic edge fusion of spiking neural networks and event-based expectation maximization for the detection and tracking of pedestrians and bicyclists. We provide a deployment-ready evaluation of the detection performance along with robustness, energy footprint, and weatherization while emphasizing the advantages of the neuro-statistical edge solution and its city-level scaling capabilities. Y1 - 2025 U6 - https://doi.org/10.1088/2634-4386/adcbcb SN - 2634-4386 PB - IOP Publishing ER - TY - JOUR A1 - Axenie, Cristian T1 - Antifragile control systems in neuronal processing: a sensorimotor perspective JF - Biological Cybernetics N2 - The stability–robustness–resilience–adaptiveness continuum in neuronal processing follows a hierarchical structure that explains interactions and information processing among the different time scales. Interestingly, using “canonical” neuronal computational circuits, such as Homeostatic Activity Regulation, Winner-Take-All, and Hebbian Temporal Correlation Learning, one can extend the behavior spectrum towards antifragility. Cast already in both probability theory and dynamical systems, antifragility can explain and define the interesting interplay among neural circuits, found, for instance, in sensorimotor control in the face of uncertainty and volatility. This perspective proposes a new framework to analyze and describe closed-loop neuronal processing using principles of antifragility, targeting sensorimotor control. Our objective is two-fold. First, we introduce antifragile control as a conceptual framework to quantify closed-loop neuronal network behaviors that gain from uncertainty and volatility. Second, we introduce neuronal network design principles, opening the path to neuromorphic implementations and transfer to technical systems. Y1 - 2025 U6 - https://doi.org/10.1007/s00422-025-01003-7 SN - 1432-0770 VL - 119 IS - 2-3 PB - Springer Nature ER - TY - CHAP A1 - Braun, Franziska A1 - Förstel, Markus A1 - Oppermann, Bastian A1 - Erzigkeit, Andreas A1 - Lehfeld, Hartmut A1 - Hillemacher, Thomas A1 - Riedhammer, Korbinian T1 - Automated Evaluation of Standardized Dementia Screening Tests T2 - Interspeech 2022 N2 - For dementia screening and monitoring, standardized tests play a key role in clinical routine since they aim at minimizing subjectivity by measuring performance on a variety of cognitive tasks. In this paper, we report a study consisting of a semistandardized history taking followed by two standardized neuropsychological tests, namely the SKT and the CERAD-NB. The tests include basic tasks such as naming objects, learning word lists, but also widely used tools such as the MMSE. Most of the tasks are performed verbally and should thus be suitable for automated scoring based on transcripts. For the first batch of 30 patients, we analyze the correlation between expert manual evaluations and automatic evaluations based on manual and automatic transcriptions. For both SKT and CERAD-NB, we observe high to perfect correlations using manual transcripts; for certain tasks with lower correlation, the automatic scoring is stricter than the human reference since it is limited to the audio. Using automatic transcriptions, correlations drop as expected and are related to recognition accuracy; however, we still observe high correlations of up to 0.98 (SKT) and 0.85 (CERADNB). We show that using word alternatives helps to mitigate recognition errors and subsequently improves correlation with expert scores. KW - dementia screening KW - neuropsychological tests Y1 - 2022 U6 - https://doi.org/10.21437/Interspeech.2022-10436 SP - 2478 EP - 2482 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Albrecht, Jens A1 - Fiedler, Marc A1 - Kiefer, Tim T1 - A Rule-Based Approach to Local Anonymization for Exclusivity Handling in Statistical Databases T2 - In: Domingo-Ferrer, J., Pejić-Bach, M. (eds) Privacy in Statistical Databases. PSD 2016. Lecture Notes in Computer Science, vol 9867. N2 - Statistical databases in general and data warehouses in particular are used to analyze large amounts of business data in pre-defined as well as ad-hoc reports. Operators of statistical databases must ensure that individual sensitive data, e.g., personal data, medical data, or business-critical data, are not revealed to unprivileged users while making use of these data in aggregates. Business rules must be defined and enforced to prevent disclosure. The unsupervised nature of ad-hoc reports, defined by the user and unknown to the database operator upfront, adds to the complexity of keeping data secure. Storing sensitive data in statistical databases demands automated methods to prevent direct or indirect disclosure of such sensitive data. This document describes a rule-based approach to local recoding of sensitive data. It introduces the notion of exclusivity to describe quasiidentifiers with local rules based on the multidimensional data model. It further defines options to treat exclusive entities that may disclose sensitive data. The local anonymization minimizes information loss and works well even with very large data sets. Y1 - 2016 U6 - https://doi.org/10.1007/978-3-319-45381-1_7 PB - Springer CY - Cham ER - TY - CHAP A1 - Gheorghe, Lisca A1 - Matthias, Kindler A1 - Tanja, Lerchl A1 - Axenie, Cristian A1 - Thomas, Grauschopf A1 - Veit, Senner T1 - The potential of reinforcement learning for lumbar load prediction in multi body models of the spine N2 - Multi body models (MBS) of the spine are an integral part of clinical and biomechanical research. Their noninvasive and adaptive character makes them a promising tool to address a large variety of questions regarding spinal loading, its causes, and consequences for the healthy and pathological spine. In sports science and athletic training, the predictive simulations of human movement can explain how changes in training, technique, or equipment affect performance and the body's biomechanical responses, helping athletes and coaches to make informed decisions. For solving predictive simulations optimization algorithms like collocation method are the most performant ones. The Reinforcement Learning (RL) algorithms propose a transition from optimization to learning. They formulate the problem of predictive simulation as learning to generate new data that describes human movement. This formulation enables them to leverage the generative power of the Artificial Neural Networks (ANNs), capable of learning models from high-dimensional and nonlinear movement data, and subsequently using these models to generate new ones within new boundary conditions. Most studies including MBS of the spine use a combination of inverse kinematics and optimization for muscle force and lumbar load estimation. However, these approaches either use generic assumptions for loading tasks of low complexity or require kinematic data from experimental studies. In this study, we address the following question: What is the potential of RL algorithms for predicting the spine’s joint torques during an extension? Y1 - 2024 U6 - https://doi.org/10.17028/rd.lboro.27045154.v1 ER - TY - CHAP A1 - Grandeit, Philipp A1 - Haberkern, Carolyn A1 - Lang, Maximiliane A1 - Albrecht, Jens A1 - Lehmann, Robert T1 - Using BERT for Qualitative Content Analysis in Psychosocial Online Counseling T2 - Proceedings of the Fourth Workshop on Natural Language Processing and Computational Social Science N2 - Qualitative content analysis is a systematic method commonly used in the social sciences to analyze textual data from interviews or online discussions. However, this method usually requires high expertise and manual effort because human coders need to read, interpret, and manually annotate text passages. This is especially true if the system of categories used for annotation is complex and semantically rich. Therefore, qualitative content analysis could benefit greatly from automated coding. In this work, we investigate the usage of machine learning-based text classification models for automatic coding in the area of psycho-social online counseling. We developed a system of over 50 categories to analyze counseling conversations, labeled over 10.000 text passages manually, and evaluated the performance of different machine learning-based classifiers against human coders. Y1 - 2020 U6 - https://doi.org/10.18653/v1/2020.nlpcss-1.2 SP - 11 EP - 23 PB - Association for Computational Linguistics CY - Stroudsburg, PA, USA ER - TY - CHAP A1 - Kiesler, Natalie A1 - Scholz, Ingo A1 - Albrecht, Jens A1 - Stappert, Friedhelm A1 - Wienkop, Uwe T1 - Novice Learners of Programming and Generative AI - Prior Knowledge Matters T2 - Proceedings of the 24th Koli Calling International Conference on Computing Education Research N2 - With the broad availability of Generative AI (GenAI), introductory programming education is starting to change. At Nuremberg Tech, we observed the doubling of failure rates to approximately 50% in the first semester course “Procedural Programming” across students of all study programs. Due to these exam results in winter 2023/24, we conducted a pilot study to gather students’ use of GenAI tools, their exam results, and prior programming education and experience. The results imply significant differences of students’ use of GenAI tools depending on their prior programming education. We will therefore extend the investigation in winter term 2024/25. Y1 - 2024 U6 - https://doi.org/10.1145/3699538.3699580 SP - 1 EP - 2 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Faria, Arlo A1 - Janin, Adam A1 - Adkoli, Sidhi A1 - Riedhammer, Korbinian T1 - Toward Zero Oracle Word Error Rate on the Switchboard Benchmark T2 - Interspeech 2022 N2 - The “Switchboard benchmark” is a very well-known test set in automatic speech recognition (ASR) research, establishing record-setting performance for systems that claim human-level transcription accuracy. This work highlights lesser-known practical considerations of this evaluation, demonstrating major improvements in word error rate (WER) by correcting the reference transcriptions and deviating from the official scoring methodology. In this more detailed and reproducible scheme, even commercial ASR systems can score below 5% WER and the established record for a research system is lowered to 2.3%. An alternative metric of transcript precision is proposed, which does not penalize deletions and appears to be more discriminating for human vs. machine performance. While commercial ASR systems are still below this threshold, a research system is shown to clearly surpass the accuracy of commercial human speech recognition. This work also explores using standardized scoring tools to compute oracle WER by selecting the best among a list of alternatives. A phrase alternatives representation is compared to utterance-level N-best lists and word-level data structures; using dense lattices and adding out-of-vocabulary words, this achieves an oracle WER of 0.18%. KW - ASR evaluation KW - Switchboard benchmark KW - oracle word error rate KW - N-best lists KW - phrase alternatives Y1 - 2022 U6 - https://doi.org/10.21437/Interspeech.2022-10959 SP - 3973 EP - 3977 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Ranzenberger, Thomas A1 - Freier, Carolin A1 - Reinold, Luca A1 - Riedhammer, Korbinian A1 - Schneider, Fabian A1 - Simic, Christopher A1 - Simon, Claudia A1 - Freisinger, Steffen A1 - Georges, Munir A1 - Bocklet, Tobias T1 - A Multidisciplinary Approach to AI-based self-motivated Learning and Teaching with Large Language Models T2 - Proceedings of DELFI Workshops 2024 N2 - We present a learning experience platform that uses machine learning methods to support students and lecturers in self-motivated online learning and teaching processes. The platform is being developed as an agile open-source collaborative project supported by multiple universities and partners. The development is guided didactically, reviewed, and scientifically evaluated in several cycles. Transparency, data protection and the copyright compliant use of the system is a central part of the project. The system further employs large language models (LLMs). Due to privacy concerns, we utilize locally hosted LLM instances and explicitly do not rely on available cloud products. Students and lecturers can interact with an LLM-based chatbot in the current prototype. The AI-generated outputs contain cross-references to the current educational video’s context, indicating if sections are based on the lectures context or world knowledge. We present the prototype and results of our qualitative evaluation from the perspective of lecturers and students. KW - Artificial Intelligence in Education KW - Learning Experience Platform KW - Open Source Software KW - Large Language Models Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024_11 PB - Gesellschaft für Informatik e.V. ER - TY - CHAP A1 - Braun, Franziska A1 - Bayerl, Sebastian A1 - Hönig, Florian A1 - Lehfeld, Hartmut A1 - Hillemacher, Thomas A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Infusing Acoustic Pause Context into Text-Based Dementia Assessment N2 - Speech pauses, alongside content and structure, offer a valuable and non-invasive biomarker for detecting dementia. This work investigates the use of pause-enriched transcripts in transformer-based language models to differentiate the cognitive states of subjects with no cognitive impairment, mild cognitive impairment, and Alzheimer’s dementia based on their speech from a clinical assessment. We address three binary classification tasks: Onset, monitoring, and dementia exclusion. The performance is evaluated through experiments on a German Verbal Fluency Test and a Picture Description Test, comparing the model’s effectiveness across different speech production contexts. Starting from a textual baseline, we investigate the effect of incorporation of pause information and acoustic context. We show the test should be chosen depending on the task, and similarly, lexical pause information and acoustic cross-attention contribute differently. KW - speech biomarkers KW - dementia assessment KW - neuropsychological tests KW - pathological speech Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2496 SN - 2958-1796 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Ranzenberger, Thomas A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Personalizing Large Sequence-to-Sequence Speech Foundation Models With Speaker Representations N2 - We present a method to personalize large transformer-based encoderdecoder speech foundation models without the need for changes in the underlying model structure or training from scratch. This is achieved by projecting speaker-specific information into the latent space of the transformer decoder via a small neural network and learning to process the speaker information along with domainspecific information via parameter-efficient finetuning. We use this method to improve the automatic speech recognition results of spoken academic German and English. Our approach yields average relative word error rate (WER) improvements of approximately 29% on German academic speech and 25% on English academic speech. It also translates well to conversational speech, achieving relative WER improvements of up to 36%, and demonstrates modest gains of up to 5% on read speech. Moreover, we observe that incorporating utterances from the recent past as personalization context yields the most significant overall improvements and that changes in voice characteristics resulting from prolonged speaking have a minimal effect on the personalization quality of academic lectures. Y1 - 2024 U6 - https://doi.org/10.1109/SLT61566.2024.10832252 ER - TY - JOUR A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Bocklet, Tobias ED - Baumann, Ilja T1 - Generative adversarial networks for whispered to voiced speech conversion: a comparative study JF - International Journal of Speech Technology N2 - Generative Adversarial Networks (GANs) have demonstrated promising results as end-to-end models for whispered to voiced speech conversion. Leveraging non-autoregressive systems like GANs capable of performing conditional waveform generation eliminates the need for separate models to estimate voiced speech features, and leads to faster inference compared to autoregressive methods. This study aims to identify the optimal GAN architecture for the whispered to voiced speech conversion task by comparing six state-of-the-art models. Furthermore, we present a method for evaluating the preservation of speaker identity and local accent, using embeddings obtained from speaker- and language identification systems. Our experimental results show that building the speech conversion system based on the HiFi-GAN architecture yields the best objective evaluation scores, outperforming the baseline by ∼9% relative using frequency-weighted Signal-to-Noise Ratio and Log Likelihood Ratio, as well as by ∼29% relative using Root Mean Squared Error. In subjective tests, HiFi-GAN yielded a mean opinion score of 2.9, significantly outperforming the baseline with a score of 1.4. Furthermore, HiFi-GAN enhanced ASR performance and preserved speaker identity and accent, with correct language detection rates of up to ∼98%. KW - Speech conversion · Generative adversarial networks · Whispered speech · Voiced speech Y1 - 2024 U6 - https://doi.org/10.1007/s10772-024-10161-1 VL - 27 ER -