TY - CHAP A1 - Braun, Franziska A1 - Bayerl, Sebastian P. A1 - Pérez-Toro, Paula A. A1 - Hönig, Florian A1 - Lehfeld, Hartmut A1 - Hillemacher, Thomas A1 - Nöth, Elmar A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Classifying Dementia in the Presence of Depression BT - A Cross-Corpus Study N2 - Automated dementia screening enables early detection and intervention, reducing costs to healthcare systems and increasing quality of life for those affected. Depression has shared symptoms with dementia, adding complexity to diagnoses. The research focus so far has been on binary classification of dementia (DEM) and healthy controls (HC) using speech from picture description tests from a single dataset. In this work, we apply established baseline systems to discriminate cognitive impairment in speech from the semantic Verbal Fluency Test and the Boston Naming Test using text, audio and emotion embeddings in a 3-class classification problem (HC vs. MCI vs. DEM). We perform cross-corpus and mixed-corpus experiments on two independently recorded German datasets to investigate generalization to larger populations and different recording conditions. In a detailed error analysis, we look at depression as a secondary diagnosis to understand what our classifiers actually learn. Y1 - 2023 U6 - https://doi.org/10.48550/arXiv.2308.08306 ER - TY - CHAP A1 - Riedhammer, Korbinian A1 - Baumann, Ilja A1 - Bayerl, Sebastian P. A1 - Bocklet, Tobias A1 - Braun, Franziska A1 - Wagner’, Dominik T1 - Medical Speech Processing for Diagnosis and Monitoring BT - Clinical Use Cases N2 - In recent years, speech processing for medical applications got significant traction. While pioneering work in the 1990ies focused on processing sustained vowels or isolated utterances, work in the 2000s already showed, that speech recognition systems, prosodic analysis and natural language processing be used to assess a large variety of speech pathologies.Here, we give an overview of how to classify selected speech pathologies including stuttering, language development, speech intelligibility after surgery, dementia and Alzheimers, depression and state-of-mind. While each of those poses a rather well-defined problem in a lab setting, we discuss the issues when integrating such methods in a clinical workflow such as diagnosis or monitoring. Starting from the question if such detectors can be used for general screening or rather as a specialist's tool, we explore the legal and privacy-related implications: patient-doctor conversations, working with children or demented seniors, bias towards examiner or patient, on-device vs. cloud processing.We conclude with a set of open questions that should be addressed to help bringing all this research from the lab to routine clinical use. Y1 - 2023 SP - 1417 EP - 1420 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Braun, Franziska A1 - Bayerl, Sebastian P. A1 - Nöth, Elmar A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Multi-class Detection of Pathological Speech with Latent Features BT - How does it perform on unseen data? N2 - The detection of pathologies from speech features is usually defined as a binary classification task with one class representing a specific pathology and the other class representing healthy speech. In this work, we train neural networks, large margin classifiers, and tree boosting machines to distinguish between four pathologies: Parkinson's disease, laryngeal cancer, cleft lip and palate, and oral squamous cell carcinoma. We show that latent representations extracted at different layers of a pre-trained wav2vec 2.0 system can be effectively used to classify these types of pathological voices. We evaluate the robustness of our classifiers by adding room impulse responses to the test data and by applying them to unseen speech corpora. Our approach achieves unweighted average F1-Scores between 74.1% and 97.0%, depending on the model and the noise conditions used. The systems generalize and perform well on unseen data of healthy speakers sampled from a variety of different sources. Y1 - 2023 U6 - https://doi.org/10.21437/Interspeech.2023-464 SN - 2958-1796 SP - 2318 EP - 2322 ER - TY - CHAP A1 - Chen, Wenda A1 - Huang, Jonathan A1 - Bocklet, Tobias T1 - Length-and Noise-aware Training Techniques for Short-utterance Speaker Recognition N2 - Speaker recognition performance has been greatly improved with the emergence of deep learning. Deep neural networks show the capacity to effectively deal with impacts of noise and reverberation, making them attractive to far-field speaker recognition systems. The x-vector framework is a popular choice for generating speaker embeddings in recent literature due to its robust training mechanism and excellent performance in various test sets. In this paper, we start with early work on including invariant representation learning (IRL) to the loss function and modify the approach with centroid alignment (CA) and length variability cost (LVC) techniques to further improve robustness in noisy, far-field applications. This work mainly focuses on improvements for short-duration test utterances (1-8s). We also present improved results on long-duration tasks. In addition, this work discusses a novel self-attention mechanism. On the VOiCES far-field corpus, the combination of the proposed techniques achieves relative improvements of 7.0% for extremely short and 8.2% for full-duration test utterances on equal error rate (EER) over our baseline system. KW - speaker recognition, invariant representation learning, centroid alignment, x-vector, far-field Y1 - 2020 U6 - https://doi.org/10.48550/arXiv.2008.12218 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Bayerl, Sebastian P. A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Speaker Adaptation for End-To-End Speech Recognition Systems in Noisy Environments N2 - We analyze the impact of speaker adaptation in end-to-end automatic speech recognition models based on transformers and wav2vec 2.0 under different noise conditions. By including speaker embeddings obtained from x-vector and ECAPA-TDNN systems, as well as i-vectors, we achieve relative word error rate improvements of up to 16.3% on LibriSpeech and up to 14.5% on Switchboard. We show that the proven method of concatenating speaker vectors to the acoustic features and supplying them as auxiliary model inputs remains a viable option to increase the robustness of end-to-end architectures. The effect on transformer models is stronger, when more noise is added to the input speech. The most substantial benefits for systems based on wav2vec 2.0 are achieved under moderate or no noise conditions. Both x-vectors and ECAPA-TDNN embeddings outperform i-vectors as speaker representations. The optimal embedding size depends on the dataset and also varies with the noise condition. KW - speaker adaptation, automatic speech recognition, end-to-end systems, transformer, wav2vec 2.0 Y1 - 2023 U6 - https://doi.org/10.48550/arXiv.2211.08774 ER - TY - CHAP A1 - Wagner, Dominik A1 - Bayerl, Sebastian P. A1 - Bocklet, Tobias A1 - Draxler, Christoph T1 - Implementing Easy-to-Use Recipes for the Switchboard Benchmark N2 - We report on our contribution of templates for tokenization, language modeling, and automatic speech recognition (ASR) on the Switchboard benchmark to the open-source general-purpose toolkit SpeechBrain. Three recipes for the training of end-to-end ASR systems were implemented. We describe their model architectures, as well as the necessary data preparation steps. The word error rates achievable with our models are comparable to or better than those of other popular toolkits. Pre-trained ASR models were made available on HuggingFace. They can be easily integrated into research projects or used directly for quick inference via a hosted inference API. Y1 - 2023 SP - 150 EP - 157 PB - TUDpress, Dresden ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Riedhammer, Korbinian A1 - Nöth, Elmar A1 - Bocklet, Tobias T1 - Detection of Vowel Errors in Children’s Speech Using Synthetic Phonetic Transcripts T2 - 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU), Taipei, Taiwan, 2023 N2 - The analysis of phonological processes is crucial in evaluating speech development disorders in children, but encounters challenges due to limited children audio data. This work focuses on automatic vowel error detection using a two-stage pipeline. The first stage uses a fine-tuned cross-lingual phone recognizer (wav2vec 2.0) to extract phone sequences from audio. The second stage employs a language model (BERT) for classification from a phone sequence, entirely trained on synthetic transcripts, to counteract the very broad range of potential mistakes. We evaluate the system on nonword audio recordings recited by preschool children from a speech development test. The results show that the classifier trained on synthetic data performs well, but its efficacy relies on the quality of the phone recognizer. The best classifier achieves an 94.7% F1 score when evaluated against phonetic ground truths, whereas the F1 score is 76.2% when using automatically recognized phone sequences. KW - Conferences ; Pipelines ; Phonetics ; Audio recording ; Speech processing ; Synthetic data ; Automatic speech recognition ; children’s speech ; vowel errors ; nonwords Y1 - 2023 U6 - https://doi.org/10.1109/ASRU57964.2023.10389704 SP - 1 EP - 8 ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Bayerl, Sebastian P. A1 - Bocklet, Tobias T1 - Nonwords Pronunciation Classification in Language Development Tests for Preschool Children T2 - Interspeech 2022 : 18-22 September 2022, Incheon, Korea N2 - This work aims to automatically evaluate whether the language development of children is age-appropriate. Validated speech and language tests are used for this purpose to test the auditory memory. In this work, the task is to determine whether spoken nonwords have been uttered correctly. We compare different approaches that are motivated to model specific language structures: Low-level features (FFT), speaker embeddings (ECAPA-TDNN), grapheme-motivated embeddings (wav2vec 2.0), and phonetic embeddings in form of senones (ASR acoustic model). Each of the approaches provides input for VGG-like 5-layer CNN classifiers. We also examine the adaptation per non-word. The evaluation of the proposed systems was performed using recordings from different kindergartens of spoken non- words. ECAPA-TDNN and low-level FFT features do not explicitly model phonetic information; wav2vec2.0 is trained on grapheme labels, our ASR acoustic model features contain (sub-)phonetic information. We found that the more granular the phonetic modeling is, the higher are the achieved recognition rates. The best system trained on ASR acoustic model features with VTLN achieved an accuracy of 89.4% and an area under the ROC (Receiver Operating Characteristic) curve (AUC) of 0.923. This corresponds to an improvement in accuracy of 20.2% and AUC of 0.309 relative compared to the FFT-baseline KW - children’s speech, speech assessment, transfer learning Y1 - 2022 U6 - https://doi.org/10.21437/Interspeech.2022-10777 SP - 3643 EP - 3647 ER - TY - CHAP A1 - Georges, Munir A1 - Huang, Jonathan A1 - Bocklet, Tobias T1 - Compact Speaker Embedding BT - lrx-vector N2 - Deep neural networks (DNN) have recently been widely used in speaker recognition systems, achieving state-of-the-art performance on various benchmarks. The x-vector architecture is especially popular in this research community, due to its excellent performance and manageable computational complexity. In this paper, we present the lrx-vector system, which is the low-rank factorized version of the x-vector embedding network. The primary objective of this topology is to further reduce the memory requirement of the speaker recognition system. We discuss the deployment of knowledge distillation for training the lrx-vector system and compare against low-rank factorization with SVD. On the VOiCES 2019 far-field corpus we were able to reduce the weights by 28% compared to the full-rank x-vector system while keeping the recognition rate constant (1.83% EER). KW - speaker recognition, x-vector, low power Y1 - 2020 U6 - https://doi.org/10.48550/arXiv.2008.05011 ER - TY - CHAP A1 - Bayerl, Sebastian P. A1 - Wagner, Dominik A1 - Nöth, Elmar A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian A1 - Sojka, Petr A1 - Kopeček, Ivan A1 - Pala, Karel A1 - Horák, Aleš T1 - The Influence of Dataset Partitioning on Dysfluency Detection Systems N2 - This paper empirically investigates the influence of different data splits and splitting strategies on the performance of dysfluency detection systems. For this, we perform experiments using wav2vec 2.0 models with a classification head as well as support vector machines (SVM) in conjunction with the features extracted from the wav2vec 2.0 model to detect dysfluencies. We train and evaluate the systems with different non-speaker-exclusive and speaker-exclusive splits of the Stuttering Events in Podcasts (SEP-28k) dataset to shed some light on the variability of results w.r.t. to the partition method used. Furthermore, we show that the SEP-28k dataset is dominated by only a few speakers, making it difficult to evaluate. To remedy this problem, we created SEP-28k-Extended (SEP-28k-E), containing semi-automatically generated speaker and gender information for the SEP-28k corpus, and suggest different data splits, each useful for evaluating other aspects of methods for dysfluency detection. KW - stuttering · dysfluencies · pathological speech · SEP-28k Y1 - 2022 U6 - https://doi.org/10.48550/arXiv.2206.03400 PB - Springer International Publishing ER - TY - CHAP A1 - Lopatka, Kuba A1 - Bocklet, Tobias T1 - State Sequence Pooling Training of Acoustic Models for Keyword Spotting T2 - Proceedings Interspeech 2020 N2 - We propose a new training method to improve HMM-based keyword spotting. The loss function is based on a score computed with the keyword/filler model from the entire input sequence. It is equivalent to max/attention pooling but is based on prior acoustic knowledge. We also employ a multi-task learning setup by predicting both LVCSR and keyword posteriors. We compare our model to a baseline trained on frame-wise cross entropy, with and without per-class weighting. We employ a low-footprint TDNN for acoustic modeling. The proposed training yields significant and consistent improvement over the baseline in adverse noise conditions. The FRR on cafeteria noise is reduced from 13.07% to 5.28% at 9 dB SNR and from 37.44% to 6.78% at 5 dB SNR. We obtain these results with only 600 unique training keyword samples. The training method is independent of the frontend and acoustic model topology. KW - keyword spotting, machine learning, speech recognition Y1 - 2020 U6 - https://doi.org/10.21437/Interspeech.2020-2722 SN - 2958-1796 SP - 4338 EP - 4342 ER - TY - CHAP A1 - Simic, Christopher A1 - Bocklet, Tobias T1 - Self-Supervised Adaptive AV Fusion Module for Pre-Trained ASR Models T2 - ICASSP 2024 - 2024 IEEE International Y1 - 2024 U6 - https://doi.org/10.1109/ICASSP48485.2024.10448047 SP - 12787 EP - 12791 ER - TY - CHAP A1 - Seeberger, Philipp A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Information Type Classification with Contrastive Task-Specialized Sentence Encoders N2 - User-generated information content has become an important information source in crisis situations. However, classification models suffer from noise and event-related biases which still poses a challenging task and requires sophisticated task-adaptation. To address these challenges, we propose the use of contrastive task-specialized sentence encoders for downstream classification. We apply the task-specialization on the CrisisLex, HumAID, and TrecIS information type classification tasks and show performance gains w.r.t. F1-score. Furthermore, we analyse the cross-corpus and cross-lingual capabilities for two German event relevancy classification datasets. Y1 - 2023 U6 - https://doi.org/10.48550/arXiv.2312.11020 PB - Association for Computational Linguistics ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Schuster, Maria A1 - Nöth, Elmar A1 - Bocklet, Tobias T1 - Towards Interpretability of Automatic Phoneme Analysis in Cleft Lip and Palate Speech T2 - IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP) N2 - Cleft Lip and Palate ranks among the most common congenital abnormalities and significantly influences speech articulation, resulting in varying phonemic impacts. In a clinical context, a detailed diagnosis is carried out by time-consuming perceptual evaluations. We use perceptual ratings of different articulatory modifications on phoneme-level as ground-truth and propose a system based on wav2vec 2.0, trained to the downstream task of classifying phonemic criteria as a multi-class and multi-label problem. The system is trained for detection on utterance level, without the usage of phoneme labels. To gain a clearer understanding of which areas of the speech signal have the greatest impact on classification, we assess the extent to which our system aligns with expert ratings at the phoneme level. Additionally, we examine which specific phonemes play a decisive role in determining the final classification of the labeled criteria. The results show that salient phonemes marked by experts contribute remarkably greater to the classification of the correct class using feature relevance explanation methods. To the best of our knowledge, this is the first study incorporating various utterance-level articulatory modifications classification and phoneme-level interpretation, offering a more comprehensive understanding for potential clinical applications. KW - pathologic speech, cleft lip and palate, children’s speech, automatic assessment Y1 - 2024 U6 - https://doi.org/10.1109/ICASSP48485.2024.10447632 SP - 12602 EP - 12606 ER - TY - CHAP A1 - Ranzenberger, Thomas A1 - Freier, Carolin A1 - Reinold, Luca A1 - Riedhammer, Korbinian A1 - Schneider, Fabian A1 - Simic, Christopher A1 - Simon, Claudia A1 - Freisinger, Steffen A1 - Georges, Munir A1 - Bocklet, Tobias T1 - A Multidisciplinary Approach to AI-based self-motivated Learning and Teaching with Large Language Models T2 - Proceedings of DELFI Workshops 2024 N2 - We present a learning experience platform that uses machine learning methods to support students and lecturers in self-motivated online learning and teaching processes. The platform is being developed as an agile open-source collaborative project supported by multiple universities and partners. The development is guided didactically, reviewed, and scientifically evaluated in several cycles. Transparency, data protection and the copyright compliant use of the system is a central part of the project. The system further employs large language models (LLMs). Due to privacy concerns, we utilize locally hosted LLM instances and explicitly do not rely on available cloud products. Students and lecturers can interact with an LLM-based chatbot in the current prototype. The AI-generated outputs contain cross-references to the current educational video’s context, indicating if sections are based on the lectures context or world knowledge. We present the prototype and results of our qualitative evaluation from the perspective of lecturers and students. KW - Artificial Intelligence in Education KW - Learning Experience Platform KW - Open Source Software KW - Large Language Models Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024_11 PB - Gesellschaft für Informatik e.V. ER - TY - CHAP A1 - Braun, Franziska A1 - Bayerl, Sebastian A1 - Hönig, Florian A1 - Lehfeld, Hartmut A1 - Hillemacher, Thomas A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Infusing Acoustic Pause Context into Text-Based Dementia Assessment N2 - Speech pauses, alongside content and structure, offer a valuable and non-invasive biomarker for detecting dementia. This work investigates the use of pause-enriched transcripts in transformer-based language models to differentiate the cognitive states of subjects with no cognitive impairment, mild cognitive impairment, and Alzheimer’s dementia based on their speech from a clinical assessment. We address three binary classification tasks: Onset, monitoring, and dementia exclusion. The performance is evaluated through experiments on a German Verbal Fluency Test and a Picture Description Test, comparing the model’s effectiveness across different speech production contexts. Starting from a textual baseline, we investigate the effect of incorporation of pause information and acoustic context. We show the test should be chosen depending on the task, and similarly, lexical pause information and acoustic cross-attention contribute differently. KW - speech biomarkers KW - dementia assessment KW - neuropsychological tests KW - pathological speech Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2496 SN - 2958-1796 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Ranzenberger, Thomas A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Personalizing Large Sequence-to-Sequence Speech Foundation Models With Speaker Representations N2 - We present a method to personalize large transformer-based encoderdecoder speech foundation models without the need for changes in the underlying model structure or training from scratch. This is achieved by projecting speaker-specific information into the latent space of the transformer decoder via a small neural network and learning to process the speaker information along with domainspecific information via parameter-efficient finetuning. We use this method to improve the automatic speech recognition results of spoken academic German and English. Our approach yields average relative word error rate (WER) improvements of approximately 29% on German academic speech and 25% on English academic speech. It also translates well to conversational speech, achieving relative WER improvements of up to 36%, and demonstrates modest gains of up to 5% on read speech. Moreover, we observe that incorporating utterances from the recent past as personalization context yields the most significant overall improvements and that changes in voice characteristics resulting from prolonged speaking have a minimal effect on the personalization quality of academic lectures. Y1 - 2024 U6 - https://doi.org/10.1109/SLT61566.2024.10832252 ER - TY - CHAP A1 - Wagner, Dominik A1 - Lee, Seanie A1 - Baumann, Ilja A1 - Seeberger, Philipp A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Optimized Speculative Sampling for {GPU} Hardware Accelerators T2 - Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing N2 - In this work, we optimize speculative sampling for parallel hardware accelerators to improve sampling speed. We notice that substantial portions of the intermediate matrices necessary for speculative sampling can be computed concurrently. This allows us to distribute the workload across multiple GPU threads, enabling simultaneous operations on matrix segments within thread blocks. This results in profiling time improvements ranging from 6% to 13% relative to the baseline implementation, without compromising accuracy. To further accelerate speculative sampling, probability distributions parameterized by softmax are approximated by sigmoid. This approximation approach results in significantly greater relative improvements in profiling time, ranging from 37% to 94%, with a minor decline in accuracy. We conduct extensive experiments on both automatic speech recognition and summarization tasks to validate the effectiveness of our optimization methods. Y1 - 2024 U6 - https://doi.org/10.18653/v1/2024.emnlp-main.370 PB - Association for Computational Linguistics CY - Miami, Florida, USA ER - TY - CHAP A1 - Ranzenberger, Thomas A1 - Bocklet, Tobias A1 - Freisinger, Steffen A1 - Georges, Munir A1 - Glockner, Kevin A1 - Herygers, Aaricia A1 - Riedhammer, Korbinian A1 - Schneider, Fabian A1 - Simic, Christopher A1 - Zakaria, Khabbab T1 - EXTENDING HANS: LARGE LANGUAGE MODELS FOR QUESTION ANSWERING, SUMMARIZATION, AND TOPIC SEGMENTATION IN AN ML-BASED LEARNING EXPERIENCE PLATFORM T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - Abstract: The use of chatbots based on large language models (LLMs) and their impact on society are influencing our learning experience platform Hochschul Assistenz-System (HAnS). HAnS uses machine learning (ML) methods to support students and lecturers in the online learning and teaching processes [1]. This paper introduces LLM-based features available in HAnS which are using the transcript of our improved Automatic Speech Recognition (ASR) pipeline with an average transcription duration of 45 seconds and an average word error rate (WER) of 6.66% on over 8 hours of audio data of 7 lecture videos. A LLM-based chatbot could be used to answer questions on the lecture content as the ASR transcript is provided as context. The summarization and topic segmentation uses the LLM to improve our learning experience platform. We generate multiple choice questions using the LLM and the ASR transcript as context during playback in a period of 3 minutes and display them in the HAnS frontend Y1 - 2024 SN - 978-3-95908-325-6 PB - TUPress CY - Dresden ER - TY - CHAP A1 - Perez-Toro, P. A. A1 - Vasquez-Correa, J. C. A1 - Arias-Vergara, T. A1 - Klumpp, P. A1 - Sierra-Castrillon, M. A1 - Roldan-Lopez, M. E. A1 - Aguillon, D. A1 - Hincapie-Henao, L. A1 - Tobon-Quintero, C. A. A1 - Bocklet, Tobias A1 - Schuster, M. A1 - Orozco-Arroyave, J. R. A1 - Nöth, E. T1 - Acoustic and Linguistic Analyses to Assess Early-Onset and Genetic Alzheimer’s Disease T2 - ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) N2 - The PSEN1-E280A or Paisa mutation is responsible for most of Early-Onset Alzheimer’s (EOA) disease cases in Colombia. It affects a large kindred of over 5000 members that present the same phenotype. The most common symptoms are related to language disorders, where speech fluency is also affected due to the difficulty to access semantic information intentionally. This study proposes the use of acoustic and linguistic methods to extract features from speech recordings and their transcriptions to discriminate people with conditions related to the Paisa mutation. We consider state-of-the-art word-embedding methods like Word2Vec and Bidirectional Encoder Representations from Transformer to process the transcripts. The speech signals are modeled by using traditional acoustic features and speaker embeddings. To the best of our knowledge, this is the first study focused on evaluating genetic Alzheimer’s and EOA using acoustics and linguistics. KW - PSEN1–E280A KW - Alzheimer’s Disease KW - Acoustic Analysis KW - Linguistic Analysis Y1 - 2021 SN - 978-1-7281-7605-5 U6 - https://doi.org/10.1109/ICASSP39728.2021.9414009 SP - 8338 EP - 8342 PB - IEEE ER - TY - CHAP A1 - Scheuerer, Ralph A1 - Haderlein, Tino A1 - Nöth, Elmar A1 - Bocklet, Tobias T1 - Applying X-Vectors on Pathological Speech After Larynx Removal T2 - 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) N2 - Speaker embeddings extracted from time delayed neural networks (TDNNs) contributed to major recent advancements in speaker recognition and verification. We use an X-Vector system trained on augmented VoxCeleb1 and VoxCeleb2 data to obtain embeddings for pathological speech after total or partial larynx removal. We show that our model is able to effectively distinguish and visualize patient groups when generating embeddings. We further compare various regression models on the task of automatically predicting different perceptual ratings by speech therapists (intelligibility, vocal effort, and overall quality) based on the extracted speaker embeddings. For both patient groups we show Pearson correlations in the range of +0.8; we find that Random Forest and Support Vector Regression produce scores that best resemble the experts' assessments. KW - laryngectomy KW - intelligibility KW - pathological speech KW - x-vectors Y1 - 2021 SN - 978-1-6654-3739-4 U6 - https://doi.org/10.1109/asru51503.2021.9688278 VL - 2021 SP - 1079 EP - 1086 PB - IEEE ER - TY - CHAP A1 - Bundscherer, Maximilian A1 - Schmitt, Thomas H. A1 - Bayerl, Sebastian P. A1 - Auerbach, Thomas A1 - Bocklet, Tobias T1 - An Acoustical Machine Learning Approach to Determine Abrasive Belt Wear of Wide Belt Sanders T2 - 2022 IEEE Sensors N2 - This paper describes a machine learning approach to determine the abrasive belt wear of wide belt sanders used in industrial processes based on acoustic data, regardless of the sanding process-related parameters, Feed speed, Grit Size, and Type of material. Our approach utilizes Decision Tree, Random Forest, k-nearest Neighbors, and Neural network Classifiers to detect the belt wear from Spectrograms, Mel Spectrograms, MFCC, IMFCC, and LFCC, yielding an accuracy of up to 86.1% on five levels of belt wear. A 96% accuracy could be achieved with different Decision Tree Classifiers specialized in different sanding parameter configurations. The classifiers could also determine with an accuracy of 97% if the machine is currently sanding or is idle and with an accuracy of 98.4% and 98.8% detect the sanding parameters Feed speed and Grit Size. We can show that low-dimensional mappings of high-dimensional features can be used to visualize belt wear and sanding parameters meaningfully. KW - Acoustic sensors KW - Abrasive belt wear KW - Tool wear KW - Machine learning KW - Industrial process KW - Wide belt sanding machines Y1 - 2022 SN - 978-1-6654-8464-0 U6 - https://doi.org/10.1109/SENSORS52175.2022.9967324 VL - 2022 PB - IEEE ER - TY - CHAP A1 - Schmitt, Thomas H. A1 - Bundscherer, Maximilian A1 - Drechsel, Ralf A1 - Bocklet, Tobias T1 - Machine learning based optimization of a ceramic bushing manufacturing process T2 - 2022 IEEE Sensors N2 - Machine learning (ML) has shown great promise in a variety of domains in recent years. ML models are known to require large amounts of labeled training data, keeping small to medium-sized business from utilizing them. This paper presents ML based approach to optimize a ceramic bushing manufac-turing process, by predicting the employed press-fit process as a function of press punch position. Accurate predictions would ensure optimal process configuration, guaranteeing quality and reducing waste. Models are trained in a supervised manner to predict the press-fit process and the ceramic defect probabilities as functions of press punch position. We were able to predict the press-fit process with a mean correlation of 0.996 and assess whether the process would damage the ceramic with a mean precision of 96.7%. Our results exemplify how ML can be used to predict and optimize highly specialised processes even with small datasets. KW - Manufacturing KW - machine learning KW - optimization KW - ceramic bushing Y1 - 2022 SN - 978-1-6654-8464-0 U6 - https://doi.org/10.1109/sensors52175.2022.9967124 PB - IEEE ER - TY - CHAP A1 - Klumpp, P. A1 - Bocklet, Tobias A1 - Arias-Vergara, T. A1 - Vásquez-Correa, J. C. A1 - Pérez-Toro, P.A. A1 - Bayerl, Sebastian P. A1 - Orozco-Arroyave, J. R. A1 - Nöth, Elmar T1 - The Phonetic Footprint of Covid-19? T2 - Interspeech 2021 N2 - Against the background of the ongoing pandemic, this year’s Computational Paralinguistics Challenge featured a classification problem to detect Covid-19 from speech recordings. The presented approach is based on a phonetic analysis of speech samples, thus it enabled us not only to discriminate between Covid and non-Covid samples, but also to better understand how the condition influenced an individual’s speech signal. Our deep acoustic model was trained with datasets collected exclusively from healthy speakers. It served as a tool for segmentation and feature extraction on the samples from the challenge dataset. Distinct patterns were found in the embeddings of phonetic classes that have their place of articulation deep inside the vocal tract. We observed profound differences in classification results for development and test splits, similar to the baseline method. We concluded that, based on our phonetic findings, it was safe to assume that our classifier was able to reliably detect a pathological condition located in the respiratory tract. However, we found no evidence to claim that the system was able to discriminate between Covid-19 and other respiratory diseases. KW - COVID-19 Y1 - 2021 U6 - https://doi.org/10.21437/Interspeech.2021-1488 SN - 2958-1796 SP - 441 EP - 445 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Bayerl, Sebastian P. A1 - Bocklet, Tobias T1 - Nonwords Pronunciation Classification in Language Development Tests for Preschool Children T2 - Interspeech 2022 N2 - This work aims to automatically evaluate whether the language development of children is age-appropriate. Validated speech and language tests are used for this purpose to test the auditory memory. In this work, the task is to determine whether spoken nonwords have been uttered correctly. We compare different approaches that are motivated to model specific language structures: Low-level features (FFT), speaker embeddings (ECAPA-TDNN), grapheme-motivated embeddings (wav2vec 2.0), and phonetic embeddings in form of senones (ASR acoustic model). Each of the approaches provides input for VGG-like 5-layer CNN classifiers. We also examine the adaptation per nonword. The evaluation of the proposed systems was performed using recordings from different kindergartens of spoken nonwords. ECAPA-TDNN and low-level FFT features do not explicitly model phonetic information; wav2vec2.0 is trained on grapheme labels, our ASR acoustic model features contain (sub-)phonetic information. We found that the more granular the phonetic modeling is, the higher are the achieved recognition rates. The best system trained on ASR acoustic model features with VTLN achieved an accuracy of 89.4% and an area under the ROC (Receiver Operating Characteristic) curve (AUC) of 0.923. This corresponds to an improvement in accuracy of 20.2% and AUC of 0.309 relative compared to the FFT-baseline. Y1 - 2022 U6 - https://doi.org/10.21437/interspeech.2022-10777 SN - 2958-1796 VL - 2022 SP - 3643 EP - 3647 PB - ISCA ER - TY - CHAP A1 - Chen, Wenda A1 - Huang, Jonathan A1 - Bocklet, Tobias T1 - Length- and Noise-Aware Training Techniques for Short-Utterance Speaker Recognition T2 - Interspeech 2020 N2 - Speaker recognition performance has been greatly improved with the emergence of deep learning. Deep neural networks show the capacity to effectively deal with impacts of noise and reverberation, making them attractive to far-field speaker recognition systems. The x-vector framework is a popular choice for generating speaker embeddings in recent literature due to its robust training mechanism and excellent performance in various test sets. In this paper, we start with early work on including invariant representation learning (IRL) to the loss function and modify the approach with centroid alignment (CA) and length variability cost (LVC) techniques to further improve robustness in noisy, far-field applications. This work mainly focuses on improvements for short-duration test utterances (1-8s). We also present improved results on long-duration tasks. In addition, this work discusses a novel self-attention mechanism. On the VOiCES far-field corpus, the combination of the proposed techniques achieves relative improvements of 7.0% for extremely short and 8.2% for full-duration test utterances on equal error rate (EER) over our baseline system. KW - speaker recognition KW - invariant representation learning KW - centroid alignment KW - x-vector KW - far-field Y1 - 2020 U6 - https://doi.org/10.21437/interspeech.2020-2872 SN - 2958-1796 SP - 3835 EP - 3839 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Wagner, Dominik A1 - Bayerl, Sebastian P. A1 - Maruri, Hector A. Cordourier A1 - Bocklet, Tobias T1 - Generative Models for Improved Naturalness, Intelligibility, and Voicing of Whispered Speech T2 - 2022 IEEE Spoken Language Technology Workshop (SLT) N2 - This work adapts two recent architectures of generative models and evaluates their effectiveness for the conversion of whispered speech to normal speech. We incorporate the normal target speech into the training criterion of vector-quantized variational autoencoders (VQ-VAEs) and Mel-GANs, thereby conditioning the systems to recover voiced speech from whispered inputs. Objective and subjective quality measures indicate that both VQ-VAEs and MelGANs can be modified to perform the conversion task. We find that the proposed approaches significantly improve the Mel cepstral distortion (MCD) metric by at least 25% relative to a Disco-GAN baseline. Subjective listening tests suggest that the MelGAN-based system significantly improves naturalness, intelligibility, and voicing compared to the whispered input speech. A novel evaluation measure based on differences between latent speech representations also indicates that our MelGAN-based approach yields improvements relative to the baseline. KW - whispered speech KW - speech conversion KW - VAE KW - GAN KW - generative models Y1 - 2023 SN - 979-8-3503-9690-4 U6 - https://doi.org/10.1109/SLT54892.2023.10022796 SP - 943 EP - 948 PB - IEEE ER - TY - CHAP A1 - Vasquez-Correa, J. C. A1 - Bocklet, Tobias A1 - Orozco-Arroyave, J. R. A1 - Nöth, E. T1 - Comparison of User Models Based on GMM-UBM and I-Vectors for Speech, Handwriting, and Gait Assessment of Parkinson’s Disease Patients T2 - ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) N2 - Parkinson's disease is a neurodegenerative disorder characterized by the presence of different motor impairments. Information from speech, handwriting, and gait signals have been considered to evaluate the neurological state of the patients. On the other hand, user models based on Gaussian mixture models - universal background models (GMMUBM) and i-vectors are considered the state-of-the-art in biometric applications like speaker verification because they are able to model specific speaker traits. This study introduces the use of GMM-UBM and i-vectors to evaluate the neurological state of Parkinson's patients using information from speech, handwriting, and gait. The results show the importance of different feature sets from each type of signal in the assessment of the neurological state of the patients. KW - Parkinson’s disease KW - GMM-UBM KW - ivectors KW - gait analysis KW - handwriting analysis KW - speech analysis Y1 - 2020 SN - 978-1-5090-6631-5 U6 - https://doi.org/10.1109/icassp40776.2020.9054348 SP - 6544 EP - 6548 PB - IEEE ER - TY - CHAP A1 - Georges, Munir A1 - Huang, Jonathan A1 - Bocklet, Tobias T1 - Compact Speaker Embedding: lrx-Vector BT - Proceedings of the Annual Conference of the International Speech Communication Association T2 - Interspeech 2020 N2 - Deep neural networks (DNN) have recently been widely used in speaker recognition systems, achieving state-of-the-art performance on various benchmarks. The x-vector architecture is especially popular in this research community, due to its excellent performance and manageable computational complexity. In this paper, we present the lrx-vector system, which is the low-rank factorized version of the x-vector embedding network. The primary objective of this topology is to further reduce the memory requirement of the speaker recognition system. We discuss the deployment of knowledge distillation for training the lrx-vector system and compare against low-rank factorization with SVD. On the VOiCES 2019 far-field corpus we were able to reduce the weights by 28% compared to the full-rank x-vector system while keeping the recognition rate constant (1.83% EER). KW - speaker recognition KW - x-vector KW - low power Y1 - 2020 U6 - https://doi.org/10.21437/interspeech.2020-2106 SN - 2958-1796 SP - 3236 EP - 3240 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Bundscherer, Maximilian A1 - Schmitt, Thomas A1 - Bocklet, Tobias T1 - Machine Learning in Industrial Quality Control of Glass Bottle Prints N2 - In industrial manufacturing of glass bottles, quality control of bottle prints is necessary as numerous factors can negatively affect the printing process. Even minor defects in the bottle prints must be detected despite reflections in the glass or manufacturing-related deviations. In cooperation with our medium-sized industrial partner, two ML-based approaches for quality control of these bottle prints were developed and evaluated, which can also be used in this challenging scenario. Our first approach utilized different filters to supress reflections (e.g. Sobel or Canny) and image quality metrics for image comparison (e.g. MSE or SSIM) as features for different supervised classification models (e.g. SVM or k-Neighbors), which resulted in an accuracy of 84%. The images were aligned based on the ORB algorithm, which allowed us to estimate the rotations of the prints, which may serve as an indicator for anomalies in the manufacturing process. In our second approach, we fine-tuned different pre-trained CNN models (e.g. ResNet or VGG) for binary classification, which resulted in an accuracy of 87%. Utilizing Grad-Cam on our fine-tuned ResNet-34, we were able to localize and visualize frequently defective bottle print regions. This method allowed us to provide insights that could be used to optimize the actual manufacturing process. This paper also describes our general approach and the challenges we encountered in practice with data collection during ongoing production, unsupervised preselection, and labeling. KW - Machine Learning, Quality Control, Industrial Manufacturing, Glass Bottle Printsuring Optimization, Glass Printing Y1 - 2024 U6 - https://doi.org/10.48550/arXiv.2409.20132 ER - TY - CHAP A1 - Wagner, Dominik A1 - Bayerl, Sebastian P. A1 - Baumann, Ilja A1 - Riedhammer, Korbinian A1 - Nöth, Elmar A1 - Bocklet, Tobias T1 - Large Language Models for Dysfluency Detection in Stuttered Speech N2 - Accurately detecting dysfluencies in spoken language can help to improve the performance of automatic speech and language processing components and support the development of more inclusive speech and language technologies. Inspired by the recent trend towards the deployment of large language models (LLMs) as universal learners and processors of non-lexical inputs, such as audio and video, we approach the task of multi-label dysfluency detection as a language modeling problem. We present hypotheses candidates generated with an automatic speech recognition system and acoustic representations extracted from an audio encoder model to an LLM, and finetune the system to predict dysfluency labels on three datasets containing English and German stuttered speech. The experimental results show that our system effectively combines acoustic and lexical information and achieves competitive results on the multi-label stuttering detection task. KW - dysfluency detection, stuttering, large language models, wav2vec 2.0, Whisper, pathological speech Y1 - 2024 U6 - https://doi.org/10.48550/arXiv.2406.11025 ER - TY - CHAP A1 - Baumann, Ilja A1 - Unger, Nicole A1 - Wagner, Dominik A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Automatic Evaluation of a Sentence Memory Test for Preschool Children N2 - Assessment of memory capabilities in preschool-aged children is crucial for early detection of potential speech development impairments or delays. We present an approach for the automatic evaluation of a standardized sentence memory test specifically for preschool children. Our methodology leverages automatic transcription of recited sentences and evaluation based on natural language processing techniques. We demonstrate the effectiveness of our approach on a dataset comprised of recited sentences from preschool-aged children, incorporating ratings of semantic and syntactic correctness. The best performing systems achieve an F1 score of 91.7% for semantic correctness and 86.1% for syntactic correctness using automatic transcripts. Our results showcase the potential of automated evaluation systems in providing reliable and efficient assessments of memory capabilities in early childhood, facilitating timely interventions and support for children with language development needs. KW - speech development, children’s speech, automatic assessment Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2125 SP - 5158 EP - 5162 ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Schuster, Maria A1 - Riedhammer, Korbinian A1 - Nöth, Elmar A1 - Bocklet, Tobias T1 - Towards Self-Attention Understanding for Automatic Articulatory Processes Analysis in Cleft Lip and Palate Speech N2 - Cleft lip and palate (CLP) speech presents unique challenges for automatic phoneme analysis due to its distinct acoustic characteristics and articulatory anomalies. We perform phoneme analysis in CLP speech using a pre-trained wav2vec 2.0 model with a multi-head self-attention classification module to capture long-range dependencies within the speech signal, thereby enabling better contextual understanding of phoneme sequences. We demonstrate the effectiveness of our approach in the classification of various articulatory processes in CLP speech. Furthermore, we investigate the interpretability of self-attention to gain insights into the model’s understanding of CLP speech characteristics. Our findings highlight the potential of the selfattention mechanisms for improving automatic phoneme analysis in CLP speech, paving the way for enhanced diagnostics, adding interpretability for therapists and affected patients. KW - pathologic speech, cleft lip and palate, children’s speech, automatic assessment Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2134 SP - 2430 EP - 2434 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias ED - Pesak, Krisztina T1 - Outlier Reduction with Gated Attention for Improved Post-training Quantization in Large Sequence-to-sequence Speech Foundation Models N2 - This paper explores the improvement of post-training quantization (PTQ) after knowledge distillation in the Whisper speech foundation model family. We address the challenge of outliers in weights and activation tensors, known to impede quantization quality in transformer-based language and vision models. Extending this observation to Whisper, we demonstrate that these outliers are also present when transformer-based models are trained to perform automatic speech recognition, necessitating mitigation strategies for PTQ. We show that outliers can be reduced by a recently proposed gating mechanism in the attention blocks of the student model, enabling effective 8-bit quantization, and lower word error rates compared to student models without the gating mechanism in place. KW - post-training quantization, Whisper, gated attention, outliers Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2105 SP - 4623 EP - 4627 ER - TY - CHAP A1 - Ranzenberger, Thomas A1 - Freier, Carolin A1 - Reinold, Luca A1 - Riedhammer, Korbinian A1 - Schneider, Fabian A1 - Simic, Christopher A1 - Simon, Claudia A1 - Freisinger, Steffen A1 - Georges, Munir A1 - Bocklet, Tobias T1 - A Multidisciplinary Approach to AI-based self-motivated Learning and Teaching with Large Language Models T2 - Proceedings of DELFI 2024 N2 - We present a learning experience platform that uses machine learning methods to support students and lecturers in self-motivated online learning and teaching processes. The platform is being developed as an agile open-source collaborative project supported by multiple universities and partners. The development is guided didactically, reviewed, and scientifically evaluated in several cycles. Transparency, data protection and the copyright compliant use of the system is a central part of the project. The system further employs large language models (LLMs). Due to privacy concerns, we utilize locally hosted LLM instances and explicitly do not rely on available cloud products. Students and lecturers can interact with an LLM-based chatbot in the current prototype. The AI-generated outputs contain cross-references to the current educational video’s context, indicating if sections are based on the lectures context or world knowledge. We present the prototype and results of our qualitative evaluation from the perspective of lecturers and students. KW - Artificial Intelligence in Education; Learning Experience Platform; Open Source Software; Large Language Models Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024_11 SN - 2944-7682 PB - Gesellschaft für Informatik e.V. CY - Bonn ER - TY - CHAP A1 - Braun, Franziska A1 - Witzl, Christopher A1 - Erzigkeit, Andreas A1 - Lehfeld, Hartmut A1 - Hillemacher, Thomas A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Pitfalls and Limits in Automatic Dementia Assessment T2 - Interspeech 2025 N2 - Current work on speech-based dementia assessment focuses on either feature extraction to predict assessment scales, or on the automation of existing test procedures. Most research uses public data unquestioningly and rarely performs a detailed error analysis, focusing primarily on numerical performance. We perform an in-depth analysis of an automated standardized dementia assessment, the Syndrom-Kurz-Test. We find that while there is a high overall correlation with human annotators, due to certain artifacts, we observe high correlations for the severely impaired individuals, which is less true for the healthy or mildly impaired ones. Speech production decreases with cognitive decline, leading to overoptimistic correlations when test scoring relies on word naming. Depending on the test design, fallback handling introduces further biases that favor certain groups. These pitfalls remain independent of group distributions in datasets and require differentiated analysis of target groups. KW - dementia screening KW - neuropsychological tests KW - pathological speech Y1 - 2025 U6 - https://doi.org/10.21437/Interspeech.2025-2280 SN - 2958-1796 SP - 5663 EP - 5667 PB - ISCA ER - TY - CHAP A1 - Bundscherer, Maximilian A1 - Schmitt, Thomas A1 - Baumann, Ilja A1 - Bocklet, Tobias T1 - Digital Operating Mode Classification of Real-World Amateur Radio Transmissions T2 - ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) N2 - This study presents an ML approach for classifying digital radio operating modes evaluated on real-world transmissions. We generated 98 different parameterized radio signals from 17 digital operating modes, transmitted each of them on the 70 cm (UHF) amateur radio band, and recorded our transmissions with two different architectures of SDR receivers. Three lightweight ML models were trained exclusively on spectrograms of limited non-transmitted signals with random characters as payloads. This training involved an online data augmentation pipeline to simulate various radio channel impairments. Our best model, EfficientNetB0, achieved an accuracy of 93.80% across the 17 operating modes and 85.47% across all 98 parameterized radio signals, evaluated on our real-world transmissions with Wikipedia articles as payloads. Furthermore, we analyzed the impact of varying signal durations & the number of FFT bins on classification, assessed the effectiveness of our simulated channel impairments, and tested our models across multiple simulated SNRs. KW - Automatic Modulation Classification KW - Amateur KW - Radio KW - Spectrum Monitoring KW - Cognitive Radio KW - Machine Learning Y1 - 2025 U6 - https://doi.org/10.1109/ICASSP49660.2025.10889837 SP - 1 EP - 5 PB - IEEE ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Optimized Self-supervised Training with BEST-RQ for Speech Recognition T2 - ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) N2 - Self-supervised learning has been successfully used for various speech related tasks, including automatic speech recognition. BERT-based Speech pre-Training with Random-projection Quantizer (BEST-RQ) has achieved state-of-the-art results in speech recognition. In this work, we further optimize the BEST-RQ approach using Kullback-Leibler divergence as an additional regularizing loss and multicodebook extension per cluster derived from low-level feature clustering. Preliminary experiments on train-100 split of LibriSpeech result in a relative improvement of 11.2% on test-clean by using multiple codebooks, utilizing a combination of cross-entropy and Kullback-Leibler divergence further reduces the word error rate by 4.5%. The proposed optimizations on full LibriSpeech pre-training and fine-tuning result in relative word error rate improvements of up to 23.8% on test-clean and 30.6% on testother using 6 codebooks. Furthermore, the proposed setup leads to faster convergence in pre-training and fine-tuning and additionally stabilizes the pre-training. KW - self-supervised learning KW - speech recognition KW - conformer KW - BEST-RQ Y1 - 2025 U6 - https://doi.org/10.1109/ICASSP49660.2025.10889362 SP - 1 EP - 5 PB - IEEE ER - TY - CHAP A1 - Freisinger, Steffen A1 - Seeberger, Philipp A1 - Ranzenberger, Thomas A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Towards Multi-Level Transcript Segmentation: LoRA Fine-Tuning for Table-of-Contents Generation T2 - Interspeech 2025 N2 - Segmenting speech transcripts into thematic sections benefits both downstream processing and users who depend on written text for accessibility. We introduce a novel approach to hierarchical topic segmentation in transcripts, generating multi-level tables of contents that capture both topic and subtopic boundaries. We compare zero-shot prompting and LoRA fine-tuning on large language models, while also exploring the integration of high-level speech pause features. Evaluations on English meeting recordings and multilingual lecture transcripts (Portuguese, German) show significant improvements over established topic segmentation baselines. Additionally, we adapt a common evaluation measure for multi-level segmentation, taking into account all hierarchical levels within one metric. KW - topic segmentation KW - spoken content segmentation KW - table of contents generation KW - hierarchical segmentation Y1 - 2025 U6 - https://doi.org/10.21437/Interspeech.2025-2792 SN - 2958-1796 SP - 276 EP - 280 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Engert, Natalie A1 - Lee, Seanie A1 - Nöth, Elmar A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Personalized Fine-Tuning with Controllable Synthetic Speech from LLM-Generated Transcripts for Dysarthric Speech Recognition T2 - Interspeech 2025 N2 - In this work, we present our submission to the Speech Accessibility Project challenge for dysarthric speech recognition. We integrate parameter-efficient fine-tuning with latent audio representations to improve an encoder-decoder ASR system. Synthetic training data is generated by fine-tuning Parler-TTS to mimic dysarthric speech, using LLM-generated prompts for corpus-consistent target transcripts. Personalization with x-vectors consistently reduces word error rates (WERs) over non-personalized fine-tuning. AdaLoRA adapters outperform full fine-tuning and standard low-rank adaptation, achieving relative WER reductions of ∼23% and ∼22%, respectively. Further improvements (∼5% WER reduction) come from incorporating wav2vec 2.0-based audio representations. Training with synthetic dysarthric speech yields up to ∼7% relative WER improvement over personalized fine-tuning alone. KW - speech recognition KW - dysarthric speech KW - personalization KW - low-rank adaptation KW - synthetic data Y1 - 2025 U6 - https://doi.org/10.21437/Interspeech.2025-2155 SN - 2958-1796 SP - 3294 EP - 3298 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Pathology-Aware Speech Encoding and Data Augmentation for Dysarthric Speech Recognition T2 - Interspeech 2025 N2 - Automatic speech recognition (ASR) for pathologic speech remains a major challenge due to high variability in articulation, phonation, and prosody distortions. In this work, we propose a pathology-aware speech encoder based on BEST-RQ pre-training, which incorporates 46k hours of speech, including pathologic and atypical speech. We continue pre-training for domain adaptation and experiment with etiology-specific codebooks. We achieve a 13.2% relative word error rate (WER) improvement using the pathology-aware speech encoder with etiology-specific continued pre-training. Additionally, we examine the impact of incorporating synthetic and out-of-domain (OOD) data to further enhance ASR performance. Synthetic data reduces WER by up to 8.7%, while OOD data improves WER by 12.2%. Finally, we introduce a semantic similaritybased data augmentation technique to optimize data selection, achieving a WER improvement of up to 9.7% while minimizing the need for additional training data. KW - pathologic speech recognition KW - speech augmentation KW - self-supervised learning Y1 - 2025 U6 - https://doi.org/10.21437/Interspeech.2025-2724 SN - 2958-1796 SP - 3289 EP - 3293 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Bocklet, Tobias T1 - Vocoder-Free Non-parallel Conversion of Whispered Speech With Masked Cycle-Consistent Generative Adversarial Networks N2 - Cycle-consistent generative adversarial networks have been widely used in non-parallel voice conversion (VC). Their ability to learn mappings between source and target features without relying on parallel training data eliminates the need for temporal alignments. However, most methods decouple the conversion of acoustic features from synthesizing the audio signal by using separate models for conversion and waveform synthesis. This work unifies conversion and synthesis into a single model, thereby eliminating the need for a separate vocoder. By leveraging cycle-consistent training and a self-supervised auxiliary training task, our model is able to efficiently generate converted high-quality raw audio waveforms. Subjective listening tests showed that our unified approach achieved improvements of up to 6.7% relative to the baseline in whispered VC. Mean opinion score predictions also yielded stable results in conventional VC (between 0.5% and 2.4% relative improvement). KW - voice conversion KW - generative adversarial networks KW - cycle-consistency KW - masking KW - whispered speech Y1 - 2025 SN - 978-3-032-02548-7 U6 - https://doi.org/10.1007/978-3-032-02548-7_20 PB - Springer CY - Cham ER -