Fakultät Informatik und Mathematik
Refine
Year of publication
Document Type
- conference proceeding (article) (53)
- Article (46)
- conference proceeding (volume) (10)
- Preprint (7)
- Working Paper (4)
- Report (2)
- conference proceeding (presentation, abstract) (1)
- Edited collection (1)
- Master's Thesis (1)
- Review (1)
Has Fulltext
- yes (126) (remove)
Is part of the Bibliography
- no (126)
Keywords
- Wirtschaftsinformatik (7)
- PPS (5)
- Digitalisierung (4)
- Enterprise-Resource-Planning (4)
- Maschinelles Lernen (4)
- Convergence (3)
- Smart-Meter (3)
- Adam optimizer (2)
- Artificial Intelligence (2)
- Auftragsabwicklung (2)
Institute
- Fakultät Informatik und Mathematik (126)
- Fakultät Angewandte Sozial- und Gesundheitswissenschaften (14)
- Regensburg Strategic IT Management (ReSITM) (14)
- Institut für Sozialforschung und Technikfolgenabschätzung (IST) (13)
- Labor für Technikfolgenabschätzung und Angewandte Ethik (LaTe) (12)
- Regensburg Center of Health Sciences and Technology - RCHST (12)
- Regensburg Medical Image Computing (ReMIC) (12)
- Labor Empirische Sozialforschung (11)
- Regensburg Center of Biomedical Engineering - RCBE (3)
- Fakultät Angewandte Natur- und Kulturwissenschaften (1)
Begutachtungsstatus
- peer-reviewed (53)
Der zweite ITG-Workshop „Sprachassistenten – Anwendungen, Implikationen, Entwicklungen“ fand am 5. März 2024 in Regensburg statt. Er bot eine organisatorische und inhaltliche Fortführung des ersten Workshops vor vier Jahren in Magdeburg 2020. Auch in diesem Jahr war er wieder der Konferenz Elektronische Sprachsignalverarbeitung angegliedert. Auf dem Workshop wurden vielfältige und interdisziplinäre Beiträge in eingeladenen Vorträgen und als eingereichte Poster präsentiert. Durch die gute Mischung von Beitragenden sowohl aus der Hochschullandschaft als auch aus der Industrie wurden die verschiedensten Aspekte anwendungsnah diskutiert.
One of the tasks PAULE[1, 2] solves is finding suitable control parameter (cp-)trajectories for a given target acoustic. These cp-trajectories can be used to synthesize speech with the articulatory speech synthesizer of the VocalTractLab (VTL) [3]. If the target acoustic contains substantial microphone noise or other background noises, occasionally PAULE optimizes not for the speech in the target, but for this background noises. By adding a speech/non-speech classifier to the feedback and planning-loop in PAULE this resynthesis of background noises should be mitigated. Unfortunately, the improvements were minor, which might be due to uninformative gradients of the classifier. The importance of informative gradients and the use classifiers to adapt PAULE to different tasks are explained and discussed.
There is high confidence for the hypothesis that in speech perception the cycles of a θ-oscillation segment the auditory signal into syllables [8]. Yet the functionality of the oscillator generating the θ-oscillation is unknown. We follow the finding that, within an auditory scene, speech is perceived as a stream given by temporal coherence [12]. We work with the hypotheses that the θ-oscillator is driven by temporal features providing this coherence. We propose a new temporal feature called O-distance, which detects the onset of a syllable - the starting point to of a θ-cycle–triggered by the temporal distance from to to the instance of the maximal rise of the loudness curve of the vowel. To extract to from the auditory signal, we use the statistical properties of this distance based on the C-center hypothesis [25], which predicts a close temporal relation of the onset consonants to the onset of a vowel. The statistics are derived from reference O-distance extracted from an articulatory database, where the minima and maxima of the loudness are related to maxima and minima of the lower incisor and tongue tip. To judge the quality of the O-distance extracted from the auditory signal, we regard the temporal deviation of the O-distance to the reference O-distance. Currently we achieve a mean deviation of 34ms.
This paper addresses the challenges and advancements in speech recognition for singing, a domain distinctly different from standard speech recognition. Singing encompasses unique challenges, including extensive pitch variations, diverse vocal styles, and background music interference. We explore key areas such as phoneme recognition, language identification in songs, keyword spotting, and full lyrics transcription. I will describe some of my own experiences when performing research on these tasks just as they were starting to gain traction, but will also show how recent developments in deep learning and large-scale datasets have propelled progress in this field. My goal is to illuminate the complexities of applying speech recognition to singing, evaluate current capabilities, and outline future research directions.
The use of chatbots based on large language models (LLMs) and their impact on society are influencing our learning experience platform Hochschul-Assistenz-System (HAnS). HAnS uses machine learning (ML) methods to support students and lecturers in the online learning and teaching processes [1]. This paper introduces LLM-based features available in HAnS which are using the transcript of our improved Automatic Speech Recognition (ASR) pipeline with an average transcription duration of 45 seconds and an average word error rate (WER) of 6.66% on over 8 hours of audio data of 7 lecture videos. A LLM-based chatbot could be used to answer questions on the lecture content as the ASR transcript is provided as context. The summarization and topic segmentation uses the LLM to improve our learning experience platform. We generate multiple choice questions using the LLM and the ASR transcript as context during playback in a period of 3 minutes and display them in the HAnS frontend.
In this study, we address the complex dynamics of emotional speech and
comprehensively examine the integration of rhythmic and vocal features to recognize emotional patterns. Our exploration is conducted using two German emotional corpora: VMEmo and EmoDB. Employing a combination of supervised methods (here linear discriminant analysis, LDA) and unsupervised techniques (here k-means clustering), we aim to uncover nuanced patterns within the emotional speech in these corpora. The application of LDA highlights salient patterns across different feature sets and focuses on the classification of speakers and prosodic characteristics. In addition, k-means clustering uncovers latent structures that reveal subtle mapping between emotions and speech behavior. Our results suggest that it is possible to cluster data based on prosodic behaviors that are influenced by emotional changes. Although precise mapping to the actual clusters derived from emotional labels could not be fully achieved, the results nonetheless reveal a moderate level of success in this investigation.
This study investigates the effects of speech segmentation methods on speaker recognition models, particularly with regard to the use of rhythmic feature sets. Using three automatic methods and one manual method on the German database of Kiel corpus, segmentation was performed based on the identification of vowel onsets. Subsequently rhythmic variability indices derived from these intervals were calculated and used for principal component analysis and support vector machine model in order to investigate the variation among speakers. The results underline the influence of signal segmentation methods on speaker recognition models.
PROM surveys, used to measure the effect of rehabilitation treatments, are typically filled out on paper, and often suffer from low response rates. Replacing it with a multimodal survey system, supporting touch and speech interaction, could lead to lower hurdles and therefore more data quantity. To do this, it requires task-specific training samples for the Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) to classify spoken answers into one of the standardized PROM answer options.
Due to the lack of training data for medical PROM surveys, we created augmented text samples with each answer option description, combined with different templates. To improve training capabilities, introduce a proper test set, and evaluate the ASR, we also collected 1,797 real voice samples within an empirical study. Further, we incorporate the contextual knowledge of the current question into our NLU architecture by implementing one classifier for every question scale.
Our results reveal that training with empirical data leads to better results than augmented data from templates and original answer option descriptions. Because of participant mislabeling of 33% due to the ambiguity of the task, we receive overall low NLU performances with up to 51.1% accuracy, and rank-1-accuracy up to 79.3%. We also find that our implementation of many scale-specific NLU classifiers significantly outperforms one NLU classifier for all labels, that incorporates the same contextual knowledge after the prediction, by 8 percent points.
Speech Recognition Errors in ASR Engines and Their Impact on Linguistic Analysis in Psychotherapies
(2024)
Modern intervention planning in psychotherapies may benefit from predicting process relevant psychotherapy constructs by automated speech analysis. One essential step is the extraction of relevant linguistic speech markers by ASR engines, which because of highly sensible data, work offline. We analyze transcription errors from NeMo, Whisper, and Wav2Vec2.0, focusing on their impact on linguistic markers usually requiring high quality transcripts. By utilizing part-of-speech tagging, we examine error occurrences among different word types. The Linguistic Inquiry and Word Count (LIWC) software aids in extracting markers. We highlight challenges in transcribing spontaneous speech, prevalent in therapy, and compare results with the Mozilla CommonVoice dataset, which features read speech.
This paper investigated whether predictability-based adjustments in production have listener-oriented consequences in perception. By manipulating the acoustic features of a target syllable in different predictability contexts in German, we tested 40 listeners’ perceptual preference for the manipulation. Four source words underwent acoustic modifications on the target syllable. Our results revealed a general preference for the original (unmodified) version over the modified one. However, listeners generally favored the unmodified version more when the source word had a higher predictable context compared to a less predictable one. The results showed that predictability-based adjustments have perceptual consequences and that listeners have predictability-based expectations in perception.