TY - CHAP A1 - Feitl, Selina A1 - Kreimeier, Julian A1 - Götzelmann, Timo T1 - Accessible Electrostatic Surface Haptics: Towards an Interactive Audiotactile Map Interface for People With Visual Impairments T2 - Proceedings of the 15th International Conference on PErvasive Technologies Related to Assistive Environments N2 - Tactile models, such as floor plans of a familiar or unfamiliar environment, can be helpful for people with visual impairments to grasp and interpret spatial information. Such plans are usually fabricated physically in a time-consuming process and are not interactive. This paper suggests presenting tactile floor plans using surface haptic feedback on an electrostatic display to overcome these limitations. Besides audiohaptically exploring tactile floor plans, our prototype also allows for voice interaction and demonstrates the control of smart home devices in this context. The evaluation was conducted in two stages with eight participants with visual impairments: First, it was investigated how individual rooms can be identified and assigned with electrostatic tactile feedback over a common dot matrix display. Second, the generation of a mental map when exploring an interactive detailed floor plan with several rooms was evaluated. Our results show that electrostatic haptic feedback enables people with visual impairments to recognize and understand graphic elements like rooms and a floor plan. However, this entails significantly more time for exploration and a higher cognitive workload when compared to a comparable but more expensive dot matrix display, though spatial and semantic smart home contextual information can be added through a voice interface, such as ”What is the room’s name I am touching?” or ”Turn on the power socket in this room.” Our preliminary but innovative approach reveals the initial potential of electrostatic feedback for accessibility and aims to help situate the opportunities and challenges in this context for wider dissemination. Y1 - 2022 U6 - https://doi.org/10.1145/3529190.3534781 SP - 522 EP - 531 PB - ACM CY - New York, NY, USA ER - TY - CHAP A1 - Groß, Rainer A1 - Freudenthaler, Kurt A1 - Ulrich, Thomas T1 - PROJECT-BASED LEARNING: KEY SUCCESS FACTORS AND BENEFITS FOR A TEACHING-LEARNING SETTING IN THE CONTEXT OF UNIVERSITY-INDUSTRY COOPERATION BASED ON TEN YEARS OF EXPERIENCE T2 - ICERI Proceedings N2 - Industrial companies increasingly realize that they cannot rely exclusively on their internal research and development activities for successful innovations. Targeted cooperation with universities enables them to obtain innovative and creative solutions to challenging industrial problems. Cooperations also deliver access to a pool of talents and skills, while overcoming their own operational blindness and also saving R&D costs in the long term. However, these cooperations are not a one-way street. They are also welcome at universities, as they complement the mission of practice-oriented teaching and thus contribute to applied research and the transfer of knowledge to industry. For the students, cooperative learning provides contact to industry, which can be used on the one hand for topics of final theses and on the other hand as an introduction to potential employers. For ten years now, so-called "idea competitions" have been conducted within a cooperation between Nuremberg Tech's Faculty of Computer Science and a locally-based, multinational conglomerate corporation. The idea competitions enable a didactic approach of project-based learning within a master-level course. From the beginning, the question was raised as to how the teaching-learning setting of the idea competition could be adapted to the needs and benefits of the partners involved. This setting was subject to a continuous improvement process. In addition, key success factors for the effective conduct of the course were derived. In this article, a phase-based model of the underlying course concept is presented and explained in detail. During the preparation phase of the course, the lecturers and the cooperation partner focus on finding a current, innovative project topic that is realizable for students within a master-level course. The following phase, the initialization of the project, then involves the students. In addition to the coordination of administrative subjects, the focus is above all on conveying the project assignments and the required basic and contextual knowledge. Furthermore, the student groups who compete with one another are composed. In the next phase, project work begins, with the generation of ideas by the groups. The lecturers limit themselves to a coaching role. Before transitioning to the following course phase, an additional contact with the cooperation partner takes place, with a discussion of the group exposés. During this step, the promising ideas worth pursuing are identified. These are then further developed in detail and solutions are worked out. This phase ends with the preparation of the project report, in combination with the final presentation of each group. In the final phase of the course, the first step is to critically evaluate the presentations of the groups by the lecturers. In these dry runs, the focus is on optimizing the logical sequence and the management suitability of the presentations. The highlight is then the closing event, at which the results achieved are presented and discussed in front of a jury consisting of representatives of the cooperation partner and the lecturers. The award ceremony and handing out of the certificates conclude the idea competition. In this paper, the essential experiences encountered as well as the key success factors derived and the emerging benefits gained are discussed. Finally, the application of these experiences to the current idea competition in the summer semester of 2019 is presented. KW - Project-based learning, university-industry cooperation, computer science education, idea competition, teaching-learning setting Y1 - 2019 U6 - https://doi.org/10.21125/iceri.2019.0662 SN - 2340-1095 VL - 1 SP - 2492 EP - 2499 PB - IATED ER - TY - CHAP A1 - Faria, Arlo A1 - Janin, Adam A1 - Adkoli, Sidhi A1 - Riedhammer, Korbinian T1 - Toward Zero Oracle Word Error Rate on the Switchboard Benchmark T2 - Interspeech 2022 N2 - The “Switchboard benchmark” is a very well-known test set in automatic speech recognition (ASR) research, establishing record-setting performance for systems that claim human-level transcription accuracy. This work highlights lesser-known practical considerations of this evaluation, demonstrating major improvements in word error rate (WER) by correcting the reference transcriptions and deviating from the official scoring methodology. In this more detailed and reproducible scheme, even commercial ASR systems can score below 5% WER and the established record for a research system is lowered to 2.3%. An alternative metric of transcript precision is proposed, which does not penalize deletions and appears to be more discriminating for human vs. machine performance. While commercial ASR systems are still below this threshold, a research system is shown to clearly surpass the accuracy of commercial human speech recognition. This work also explores using standardized scoring tools to compute oracle WER by selecting the best among a list of alternatives. A phrase alternatives representation is compared to utterance-level N-best lists and word-level data structures; using dense lattices and adding out-of-vocabulary words, this achieves an oracle WER of 0.18%. KW - ASR evaluation KW - Switchboard benchmark KW - oracle word error rate KW - N-best lists KW - phrase alternatives Y1 - 2022 U6 - https://doi.org/10.21437/Interspeech.2022-10959 SP - 3973 EP - 3977 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Tammewar, Aniruddha A1 - Braun, Franziska A1 - Roccabruna, Gabriel A1 - Bayerl, Sebastian P. A1 - Riedhammer, Korbinian A1 - Riccardi, Giuseppe T1 - Annotation of Valence for Spoken Personal Narratives N2 - Personal Narrative (PN) is the recollection of individuals’ life experiences, events, and thoughts along with the associated emotions in the form of a story. Compared to other genres such as social media texts or microblogs, where people write about ex-perienced events or products, the spoken PNs are complex to analyze and understand. They are usually long and unstructured, involving multiple and related events, characters as well as thoughts and emotions associated with events, objects, and persons. In spoken PNs, emotions are conveyed by changing the speech signal characteristics as well as the lexical content of the narrative. In this work, we annotate a corpus of spoken personal narratives, with the emotion valence using discrete values. The PNs are segmented into speech segments, and the annotators annotate them in the discourse context, with values on a 5 point bipolar scale ranging from -2 to +2 (0 for neutral). In this way, we capture the unfolding of the PNs events and changes in the emotional state of the narrator. We perform an in-depth analysis of the inter-annotator agreement, the relation between the label distribution w.r.t. the stimulus (positive/negative) used for the elicitation of the narrative, and compare the segment-level annotations to a baseline continuous annotation. We find that the neutral score plays an important role in the agreement. We observe that it is easy to differentiate the positive from the negative valence while the confusion with the neutral label is high. KW - Personal Narratives, Emotion Annotation, Segment Level Annotation Y1 - 2022 ER - TY - CHAP A1 - Seeberger, Philipp A1 - Riedhammer, Korbinian T1 - Combining Deep Neural Reranking and Unsupervised Extraction for Multi-Query Focused Summarization N2 - The CrisisFACTS Track aims to tackle challenges such as multi-stream fact-finding in the domain of event tracking; participants' systems extract important facts from several disaster-related events while incorporating the temporal order. We propose a combination of retrieval, reranking, and the well-known Integer Linear Programming (ILP) and Maximal Marginal Relevance (MMR) frameworks. In the former two modules, we explore various methods including an entity-based baseline, pre-trained and fine-tuned Question Answering systems, and ColBERT. We then use the latter module as an extractive summarization component by taking diversity and novelty criteria into account. The automatic scoring runs show strong results across the evaluation setups but also reveal shortcomings and challenges. Y1 - 2023 U6 - https://doi.org/10.48550/arXiv.2302.01148 PB - National Institute of Standards and Technology (NIST) ER - TY - CHAP A1 - Seeberger, Philipp A1 - Riedhammer, Korbinian T1 - Enhancing Crisis-Related Tweet Classification with Entity-Masked Language Modeling and Multi-Task Learning N2 - Social media has become an important information source for crisis management and provides quick access to ongoing developments and critical information. However, classification models suffer from event-related biases and highly imbalanced label distributions which still poses a challenging task. To address these challenges, we propose a combination of entity-masked language modeling and hierarchical multi-label classification as a multi-task learning problem. We evaluate our method on tweets from the TREC-IS dataset and show an absolute performance gain w.r.t. F1-score of up to 10% for actionable information types. Moreover, we found that entity-masking reduces the effect of overfitting to in-domain events and enables improvements in cross-event generalization. Y1 - 2022 U6 - https://doi.org/10.48550/arXiv.2211.11468 SP - 70 EP - 78 PB - Association for Computational Linguistics ER - TY - CHAP A1 - Zoppelt, Markus A1 - Tavakoli Kolagari, Ramin T1 - SAM: A Security Abstraction Model for Automotive Software Systems T2 - Lecture Notes in Computer Science Y1 - 2019 SN - 9783030168735 U6 - https://doi.org/10.1007/978-3-030-16874-2_5 SN - 0302-9743 SP - 59 EP - 74 PB - Springer International Publishing CY - Cham ER - TY - CHAP A1 - Wägemann, Tobias A1 - Tavakoli Kolagari, Ramin A1 - Schmid, Klaus T1 - ADOOPLA - Combining Product-Line- and Product-Level Criteria in Multi-objective Optimization of Product Line Architectures T2 - Lecture Notes in Computer Science Y1 - 2019 SN - 9783030299828 U6 - https://doi.org/10.1007/978-3-030-29983-5_9 SN - 0302-9743 SP - 126 EP - 142 PB - Springer International Publishing CY - Cham ER - TY - CHAP A1 - Wagemann, Tobias A1 - Tavakoli Kolagari, Ramin A1 - Schmid, Klaus T1 - Exploring Automotive Stakeholder Requirements for Architecture Optimization Support T2 - 2019 IEEE International Conference on Software Architecture Companion (ICSA-C) Y1 - 2019 U6 - https://doi.org/10.1109/ICSA-C.2019.00015 SP - 37 EP - 44 PB - IEEE ER - TY - CHAP A1 - Wägemann, Tobias A1 - Langer, Tobias A1 - Mottok, Jürgen A1 - Osinski, Lukas A1 - Stappert, Friedhelm A1 - Tavakoli Kolagari, Ramin T1 - Models for Dependable Heterogenous Multi- and Many-Core System Software Design Revisited T2 - 29th International Conference on Architecture of Computing Systems Y1 - 2016 ER - TY - CHAP A1 - Auernhammer, Katja A1 - Tavakoli Kolagari, Ramin A1 - Zoppelt, Markus T1 - Attacks on machine learning: Lurking danger for accountability T2 - Proceedings of the 2019 AAAI Workshop on Artificial Intelligence Safety, SafeAI 2019 Y1 - 2019 VL - 2301 PB - CEUR-WS ER - TY - CHAP A1 - Cuenot, Philippe A1 - Frey, Patrick A1 - Johansson, Rolf A1 - Lönn, Henrik A1 - Papadopoulos, Yiannis A1 - Reiser, Mark-Oliver A1 - Sandberg, Anders A1 - Servat, David A1 - Tavakoli Kolagari, Ramin A1 - Törngren, Martin A1 - Weber, Matthias T1 - 11 The EAST-ADL Architecture Description Language for Automotive Embedded Software T2 - Lecture Notes in Computer Science N2 - Current trends in automotive embedded systems focus on how to manage the increasing software content, with a strong emphasis on standardization of the embedded software structure. The management of engineering information remains a critical challenge in order to support development and other stages of the life-cycle. System modelling based on an Architecture Description Language (ADL) is a way to keep these assets within one information structure. This paper presents the EAST-ADL2 modelling language, developed in the ITEA EAST-EEA project and further enhanced in the ATESST project (www.atesst.org). EAST-ADL2 supports comprehensive model-based development of embedded systems and provides dedicated constructs to facilitate variability and product line management, requirements engineering, representation of functional as well as software/hardware solutions, and timing and safety analysis. Y1 - 2010 SN - 9783642162763 U6 - https://doi.org/10.1007/978-3-642-16277-0_11 SN - 0302-9743 SP - 297 EP - 307 PB - Springer Berlin Heidelberg CY - Berlin, Heidelberg ER - TY - CHAP A1 - Marutschke, Daniel Moritz A1 - Brockmann, Patricia T1 - A Conceptual Framework for Multidisciplinary Data Science Education Based on IoT Projects T2 - International Workshop on Artificial Intelligence and Education (WAIE) N2 - The necessity for professionals to make predictions using digital data has been recognized for many decades. With its roots in statistical analysis and later implemented as data analytics, Data Science emerged as a widely accepted discipline. The data science life cycle begins by setting a goal and ends with reporting/reflecting on the gained results. It incorporates known analytical techniques, including modern artificial intelligence and machine learning techniques. While often considered an independent discipline, both engineering and liberal arts colleges have recognized the need for an understanding of data science in the general workforce. The literature still suggests a disconnect between the study fields and how the Internet of Things could be leveraged in Project-Based Learning (PBL) style data science education. In this paper, the authors propose a conceptual framework to teach multidisciplinary data science at university level to engineering and liberal arts students using data from real-world Internet of Things (IoT) projects. KW - Data Science KW - Internet of Things KW - Educational Technology KW - Higher Education Y1 - 2024 U6 - https://doi.org/10.1109/WAIE63876.2024.00065 VL - 2024 SP - 331 EP - 335 PB - IEEE Xplore ER - TY - CHAP A1 - Domanski, Jan A1 - Reif, Jonas A1 - Schaab, Lea A1 - Schötteler, Sebastian A1 - Schuhbauer, Heidi ED - Sever Mališ, ED - Načinović Braje, ED - Raguž Krištić, T1 - JUDGED BY ITS COVER: UNDERSTANDING THE FORMATION OF BOOK TESTIMONIAL NETWORKS T2 - Proceedings of FEB Zagreb 16th International Odyssey Conference on Economics and Business N2 - Testimonials, short promotional statements often written by notable authors or critics, are a crucial marketing tool in the publishing industry, contributing to a book’s market success. This study investigates the formation of ties in testimonial networks, focusing on both network-endogenous factors (reciprocity, transitivity and preferential attachment) and exogenous factors tied to social homophily, such as shared publisher, genre and gender. A network of 575 authors was constructed from testimonials in fiction books published between 2020 and 2024 and analyzed using Exponential Random Graph Models (ERGM). The results reveal strong effects of reciprocity, transitivity, and homophily, particularly with respect to publisher and genre. Reciprocity appears to play a key role in testimonial exchanges, suggesting that authors who provide testimonials are significantly more likely to receive testimonials in return. Authors are likely to form connections through shared relationships. However, the network does not exhibit significant inequality in tie distribution, indicating that preferential attachment is not a dominant mechanism. The results strongly confirm that shared attributes, especially publisher and genre, significantly influence testimonial formation, with publisher-based homophily being the most impactful. This reflects the role of publishers in orchestrating testimonial exchanges, as they may actively encourage authors within their networks to provide testimonials for one another. These findings offer insights into the structural dynamics of testimonial networks and provide practical implications for emerging authors seeking to establish connections in the competitive publishing market. The strong evidence for reciprocity underscores the im- portance of actively engaging in the testimonial exchange process. The lack of evidence for preferential attachment is a promising finding for emerging authors, as it suggests they are not disadvantaged by having fewer initial connections. KW - Social Network Analysis, Testimonials, Book Market, ERGM Y1 - 2025 PB - University of Zagreb, Faculty of Economics & Business CY - Zagreb ER - TY - CHAP A1 - Ranzenberger, Thomas A1 - Freier, Carolin A1 - Reinold, Luca A1 - Riedhammer, Korbinian A1 - Schneider, Fabian A1 - Simic, Christopher A1 - Simon, Claudia A1 - Freisinger, Steffen A1 - Georges, Munir A1 - Bocklet, Tobias T1 - A Multidisciplinary Approach to AI-based self-motivated Learning and Teaching with Large Language Models T2 - Proceedings of DELFI Workshops 2024 N2 - We present a learning experience platform that uses machine learning methods to support students and lecturers in self-motivated online learning and teaching processes. The platform is being developed as an agile open-source collaborative project supported by multiple universities and partners. The development is guided didactically, reviewed, and scientifically evaluated in several cycles. Transparency, data protection and the copyright compliant use of the system is a central part of the project. The system further employs large language models (LLMs). Due to privacy concerns, we utilize locally hosted LLM instances and explicitly do not rely on available cloud products. Students and lecturers can interact with an LLM-based chatbot in the current prototype. The AI-generated outputs contain cross-references to the current educational video’s context, indicating if sections are based on the lectures context or world knowledge. We present the prototype and results of our qualitative evaluation from the perspective of lecturers and students. KW - Artificial Intelligence in Education KW - Learning Experience Platform KW - Open Source Software KW - Large Language Models Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024_11 PB - Gesellschaft für Informatik e.V. ER - TY - CHAP A1 - Braun, Franziska A1 - Bayerl, Sebastian A1 - Hönig, Florian A1 - Lehfeld, Hartmut A1 - Hillemacher, Thomas A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Infusing Acoustic Pause Context into Text-Based Dementia Assessment N2 - Speech pauses, alongside content and structure, offer a valuable and non-invasive biomarker for detecting dementia. This work investigates the use of pause-enriched transcripts in transformer-based language models to differentiate the cognitive states of subjects with no cognitive impairment, mild cognitive impairment, and Alzheimer’s dementia based on their speech from a clinical assessment. We address three binary classification tasks: Onset, monitoring, and dementia exclusion. The performance is evaluated through experiments on a German Verbal Fluency Test and a Picture Description Test, comparing the model’s effectiveness across different speech production contexts. Starting from a textual baseline, we investigate the effect of incorporation of pause information and acoustic context. We show the test should be chosen depending on the task, and similarly, lexical pause information and acoustic cross-attention contribute differently. KW - speech biomarkers KW - dementia assessment KW - neuropsychological tests KW - pathological speech Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2496 SN - 2958-1796 ER - TY - CHAP A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Ranzenberger, Thomas A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Personalizing Large Sequence-to-Sequence Speech Foundation Models With Speaker Representations N2 - We present a method to personalize large transformer-based encoderdecoder speech foundation models without the need for changes in the underlying model structure or training from scratch. This is achieved by projecting speaker-specific information into the latent space of the transformer decoder via a small neural network and learning to process the speaker information along with domainspecific information via parameter-efficient finetuning. We use this method to improve the automatic speech recognition results of spoken academic German and English. Our approach yields average relative word error rate (WER) improvements of approximately 29% on German academic speech and 25% on English academic speech. It also translates well to conversational speech, achieving relative WER improvements of up to 36%, and demonstrates modest gains of up to 5% on read speech. Moreover, we observe that incorporating utterances from the recent past as personalization context yields the most significant overall improvements and that changes in voice characteristics resulting from prolonged speaking have a minimal effect on the personalization quality of academic lectures. Y1 - 2024 U6 - https://doi.org/10.1109/SLT61566.2024.10832252 ER - TY - JOUR A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Bocklet, Tobias ED - Baumann, Ilja T1 - Generative adversarial networks for whispered to voiced speech conversion: a comparative study JF - International Journal of Speech Technology N2 - Generative Adversarial Networks (GANs) have demonstrated promising results as end-to-end models for whispered to voiced speech conversion. Leveraging non-autoregressive systems like GANs capable of performing conditional waveform generation eliminates the need for separate models to estimate voiced speech features, and leads to faster inference compared to autoregressive methods. This study aims to identify the optimal GAN architecture for the whispered to voiced speech conversion task by comparing six state-of-the-art models. Furthermore, we present a method for evaluating the preservation of speaker identity and local accent, using embeddings obtained from speaker- and language identification systems. Our experimental results show that building the speech conversion system based on the HiFi-GAN architecture yields the best objective evaluation scores, outperforming the baseline by ∼9% relative using frequency-weighted Signal-to-Noise Ratio and Log Likelihood Ratio, as well as by ∼29% relative using Root Mean Squared Error. In subjective tests, HiFi-GAN yielded a mean opinion score of 2.9, significantly outperforming the baseline with a score of 1.4. Furthermore, HiFi-GAN enhanced ASR performance and preserved speaker identity and accent, with correct language detection rates of up to ∼98%. KW - Speech conversion · Generative adversarial networks · Whispered speech · Voiced speech Y1 - 2024 U6 - https://doi.org/10.1007/s10772-024-10161-1 VL - 27 ER - TY - CHAP A1 - Seeberger, Philipp A1 - Wagner, Dominik A1 - Riedhammer, Korbinian T1 - Multimodal Multimedia Event Argument Extraction with Unified Template Filling T2 - Findings of the Association for Computational Linguistics: EMNLP 2024 N2 - With the advancement of multimedia technologies, news documents and user-generated content are often represented as multiple modalities, making Multimedia Event Extraction (MEE) an increasingly important challenge. However, recent MEE methods employ weak alignment strategies and data augmentation with simple classification models, which ignore the capabilities of natural language-formulated event templates for the challenging Event Argument Extraction (EAE) task. In this work, we focus on EAE and address this issue by introducing a unified template filling model that connects the textual and visual modalities via textual prompts. This approach enables the exploitation of cross-ontology transfer and the incorporation of event-specific semantics. Experiments on the M2E2 benchmark demonstrate the effectiveness of our approach. Our system surpasses the current SOTA on textual EAE by +7% F1, and performs generally better than the second-best systems for multimedia EAE. Y1 - 2024 U6 - https://doi.org/10.18653/v1/2024.findings-emnlp.381 PB - Association for Computational Linguistics CY - Miami, Florida, USA ER - TY - CHAP A1 - Wagner, Dominik A1 - Lee, Seanie A1 - Baumann, Ilja A1 - Seeberger, Philipp A1 - Riedhammer, Korbinian A1 - Bocklet, Tobias T1 - Optimized Speculative Sampling for {GPU} Hardware Accelerators T2 - Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing N2 - In this work, we optimize speculative sampling for parallel hardware accelerators to improve sampling speed. We notice that substantial portions of the intermediate matrices necessary for speculative sampling can be computed concurrently. This allows us to distribute the workload across multiple GPU threads, enabling simultaneous operations on matrix segments within thread blocks. This results in profiling time improvements ranging from 6% to 13% relative to the baseline implementation, without compromising accuracy. To further accelerate speculative sampling, probability distributions parameterized by softmax are approximated by sigmoid. This approximation approach results in significantly greater relative improvements in profiling time, ranging from 37% to 94%, with a minor decline in accuracy. We conduct extensive experiments on both automatic speech recognition and summarization tasks to validate the effectiveness of our optimization methods. Y1 - 2024 U6 - https://doi.org/10.18653/v1/2024.emnlp-main.370 PB - Association for Computational Linguistics CY - Miami, Florida, USA ER - TY - CHAP A1 - Bergler, Matthias A1 - Tolvanen, Juha-Pekka A1 - Tavakoli Kolagari, Ramin T1 - Integrating Security and Safety with Systems Engineering: a Model-Based Approach N2 - Development of reliable systems requires that safety and security concerns are acknowledged during system development. Adding them afterwards is risky as many concerns are missed if not elicited together with the system requirements. Unfortunately, languages for systems engineering, like SysML, typically ignore security and safety forcing development teams to split the work into different formats, languages and tools without easy collaboration, with limited traceability, separate versioning and restricted use of automation that tools can provide. We present a model-based approach targeting automotive that integrates safety and security aspects with other system development practices. This is achieved via a comprehensive domain-specific modeling language that is extendable by language users. We demonstrate this approach with practical examples on how security and safety concerns are recognized along with traditional system design and analysis phases. KW - model-based development KW - security KW - safety KW - domainspecific language KW - system engineering KW - software engineering Y1 - 2022 U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:bvb:92-opus4-14863 ER - TY - CHAP A1 - Brost, Julian A1 - Egger, Christoph A1 - Lai, Russell A1 - Schmid, Fritz A1 - Schröder, Dominique A1 - Zoppelt, Markus T1 - Threshold Password-Hardened Encryption Services N2 - Password-hardened encryption (PHE) was introduced by Lai et al. at USENIX 2018 and immediately productized by VirgilSecurity. PHE is a password-based key derivation protocol that involves an oblivious external crypto service for key derivation. The security of PHE protects against offline brute-force attacks, even when the attacker is given the entire database. Furthermore, the crypto service neither learns the derived key nor the password. PHE supports key-rotation meaning that both the server and crypto service can update their keys without involving the user. While PHE significantly strengthens data security, it introduces a single point of failure because key-derivation always requires access to the crypto service. In this work, we address this issue and simultaneously increase security by introducing threshold password-hardened encryption. Our formalization of this primitive revealed shortcomings of the original PHE definition that we also address in this work. Following the spirit of prior works, we give a simple and efficient construction using lightweight tools only. We also implement our construction and evaluate its efficiency. Our experiments confirm the practical efficiency of our scheme and show that it is more efficient than common memory-hard functions, such as scrypt. From a practical perspective this means that threshold PHE can be used as an alternative to scrypt for password protection and key-derivation, offering better security in terms of offline brute force attacks. CCS CONCEPTS • Security and privacy → Cryptography. Y1 - 2020 U6 - https://doi.org/10.1145/3372297.3417266 SP - 409 EP - 424 PB - ACM ER - TY - CHAP A1 - Bergler, Matthias A1 - Tavakoli Kolagari, Ramin A1 - Lundqvist, Kristina T1 - Case study on the use of the SafeML approach in training autonomous driving vehicles N2 - The development quality for the control software for autonomous vehicles is rapidly progressing, so that the control units in the field generally perform very reliably. Nevertheless, fatal misjudgments occasionally occur putting people at risk: such as the recent accident in which a Tesla vehicle in Autopilot mode rammed a police vehicle. Since the object recognition software which is a part of the control software is based on machine learning (ML) algorithms at its core, one can distinguish a training phase from a deployment phase of the software. In this paper we investigate to what extent the deployment phase has an impact on the robustness and reliability of the software; because just as traditional, software based on ML degrades with time. A widely known effect is the so-called concept drift: in this case, one finds that the deployment conditions in the field have changed and the software, based on the outdated training data, no longer responds adequately to the current field situation. In a previous research paper, we developed the SafeML approach with colleagues from the University of Hull, where datasets are compared for their statistical distance measures. In doing so, we detected that for simple, benchmark data, the statistical distance correlates with the classification accuracy in the field. The contribution of this paper is to analyze the applicability of the SafeML approach to complex, multidimensional data used in autonomous driving. In our analysis, we found that the SafeML approach can be used for this data as well. In practice, this would mean that a vehicle could constantly check itself and detect concept drift situation early. KW - Automotive KW - Safety KW - SafeML KW - Machine Learning KW - Autonomous Driving Y1 - 2022 ER - TY - CHAP A1 - Ranzenberger, Thomas A1 - Bocklet, Tobias A1 - Freisinger, Steffen A1 - Georges, Munir A1 - Glockner, Kevin A1 - Herygers, Aaricia A1 - Riedhammer, Korbinian A1 - Schneider, Fabian A1 - Simic, Christopher A1 - Zakaria, Khabbab T1 - EXTENDING HANS: LARGE LANGUAGE MODELS FOR QUESTION ANSWERING, SUMMARIZATION, AND TOPIC SEGMENTATION IN AN ML-BASED LEARNING EXPERIENCE PLATFORM T2 - Elektronische Sprachsignalverarbeitung 2024, Tagungsband der 35. Konferenz, Regensburg, 6.-8. März 2024 N2 - Abstract: The use of chatbots based on large language models (LLMs) and their impact on society are influencing our learning experience platform Hochschul Assistenz-System (HAnS). HAnS uses machine learning (ML) methods to support students and lecturers in the online learning and teaching processes [1]. This paper introduces LLM-based features available in HAnS which are using the transcript of our improved Automatic Speech Recognition (ASR) pipeline with an average transcription duration of 45 seconds and an average word error rate (WER) of 6.66% on over 8 hours of audio data of 7 lecture videos. A LLM-based chatbot could be used to answer questions on the lecture content as the ASR transcript is provided as context. The summarization and topic segmentation uses the LLM to improve our learning experience platform. We generate multiple choice questions using the LLM and the ASR transcript as context during playback in a period of 3 minutes and display them in the HAnS frontend Y1 - 2024 SN - 978-3-95908-325-6 PB - TUPress CY - Dresden ER - TY - CHAP A1 - Zoppelt, Markus A1 - Tavakoli Kolagari, Ramin T1 - What Today's Serious Cyber Attacks on Cars Tell Us BT - Consequences for Automotive Security and Dependability T2 - Model-Based Safety and Assessment : 6th International Symposium, IMBSA 2019, Thessaloniki, Greece, October 16–18, 2019, Proceedings N2 - Highly connected with the environment via various interfaces, cars have been the focus of malicious cyber attacks for years. These attacks are becoming an increasing burden for a society with growing vehicle autonomization: they are the sword of Damocles of future mobility. Therefore, research is particularly active in the area of vehicle IT security, and in part also in the area of dependability, in order to develop effective countermeasures and to maintain a minimum of one step ahead of hackers. This paper examines the known state-of-the-art security and dependability measures based on a detailed and systematic analysis of published cyber attacks on automotive software systems. The sobering result of the analysis of the cyber attacks with the model-based technique SAM (Security Abstraction Model) and a categorization of the examined attacks in relation to the known security and dependability measures is that most countermeasures against cyber attacks are hardly effective. They either are not applicable to the underlying problem or take effect too late; the intruder has already gained access to a substantial part of the vehicle when the countermeasures apply. The paper is thus contributing to an understanding of the gaps that exist today in the area of vehicle security and dependability and concludes concrete research challenges. KW - Automotive Security KW - Automotive System Architecture KW - Dependability KW - Model-Driven Engineering Methodologies Y1 - 2019 SN - 978-3-030-32872-6 U6 - https://doi.org/10.1007/978-3-030-32872-6_18 SP - 219 EP - 285 PB - Springer CY - Cham ER - TY - CHAP A1 - Zoppelt, Markus A1 - Tavakoli Kolagari, Ramin T1 - UnCle SAM: Modeling Cloud Attacks with the Automotive Security Abstraction Model T2 - CLOUD COMPUTING 2019 : The Tenth International Conference on Cloud Computing, GRIDs, and Virtualization N2 - Driverless (autonomous) vehicles will have greater attack potential than any other individual mobility vehicles ever before. Most intelligent vehicles require communication interfaces to the environment, direct connections (e.g., Vehicle-to-X (V2X)) to an Original Equipment Manufacturer (OEM) backend service or a cloud. By connecting to the Internet, which is not only necessary for the infotainment systems, cars could increasingly turn into targets for malware or botnet attacks. Remote control via the Internet by a remote attacker is also conceivable, as has already been impressively demonstrated. This paper examines security modeling for cloud-based remote attacks on autonomous vehicles using a Security Abstraction Model (SAM) for automotive software systems). SAM adds to the early phases of (automotive) software architecture development by explicitly documenting attacks and handling them with security techniques. SAM also provides the basis for comprehensive security analysis techniques, such as the already available Common Vulnerability Scoring System (CVSS) or any other attack assessment system. KW - Automotive Security KW - Automotive Software Engineering KW - Security Modeling KW - Cloud Attacks KW - OTA Updates Y1 - 2019 SN - 978-1-61208-703-0 ER - TY - CHAP A1 - Wägemann, Tobias A1 - Werner, Albert T1 - Generating Multi-objective Programs from Variant-rich EAST-ADL Product Line Architectures T2 - INFORMATIK 2015 N2 - The design of robust architectures for software-intensive systems in the automotive domain is a complex task and a considerable engineering challenge. Even conventional systems are usually subject to a multitude of conflicting design goals, such as unit cost and weight minimization, dependability augmentation and timing control. One feasible approach to resolve these conflicting levers on a system's architecture is to perform an optimization analysis on a concretely defined design space, which in general is vast. For the purpose of the approach presented here, this design space is represented by an automated identification of variation points relevant for design space exploration. The identified variability information is then transformed into a convenient mathematical representation for product-line-aware architecture optimization. KW - Model-based Analysis KW - Multi-objective Programming KW - Architecture Optimization KW - Pareto Optimality KW - Product Line Engineering Y1 - 2015 SN - 978-3-88579-640-4 SN - 1617-5468 SP - 1673 EP - 1685 PB - Gesellschaft für Informatik e.V.. CY - Bonn ER - TY - JOUR A1 - Pérez-Toro, Paula Andrea A1 - Vásquez-Correa, Juan Camilo A1 - Bocklet, Tobias A1 - Nöth, Elmar A1 - Orozco-Arroyave, Juan Rafael T1 - User State Modeling Based on the Arousal-Valence Plane: Applications in Customer Satisfaction and Health-Care JF - IEEE Transactions on Affective Computing N2 - The acoustic analysis helps to discriminate emotions according to non-verbal information, while linguistics aims to capture verbal information from written sources. Acoustic and linguistic analyses can be addressed for different applications, where information related to emotions, mood, or affect are involved. The Arousal-Valence plane is commonly used to model emotional states in a multidimensional space. This study proposes a methodology focused on modeling the user’s state based on the Arousal-Valence plane in different scenarios. Acoustic and linguistic information are used as input to feed different deep learning architectures mainly based on convolutional and recurrent neural networks, which are trained to model the Arousal-Valence plane. The proposed approach is used for the evaluation of customer satisfaction in call-centers and for health-care applications in the assessment of depression in Parkinson’s disease and the discrimination of Alzheimer’s disease. F-scores of up to 0.89 are obtained for customer satisfaction, of up to 0.82 for depression in Parkinson’s patients, and of up to 0.80 for Alzheimer’s patients. The proposed approach confirms that there is information embedded in the Arousal-Valence plane that can be used for different purposes. KW - Arousal-valence plane KW - acoustic KW - linguistic KW - customer satisfaction KW - alzheimer’s disease KW - depression Y1 - 2021 U6 - https://doi.org/10.1109/taffc.2021.3112543 SN - 1949-3045 VL - 14 IS - 2 SP - 1533 EP - 1546 PB - Institute of Electrical and Electronics Engineers (IEEE) ER - TY - CHAP A1 - Perez-Toro, P. A. A1 - Vasquez-Correa, J. C. A1 - Arias-Vergara, T. A1 - Klumpp, P. A1 - Sierra-Castrillon, M. A1 - Roldan-Lopez, M. E. A1 - Aguillon, D. A1 - Hincapie-Henao, L. A1 - Tobon-Quintero, C. A. A1 - Bocklet, Tobias A1 - Schuster, M. A1 - Orozco-Arroyave, J. R. A1 - Nöth, E. T1 - Acoustic and Linguistic Analyses to Assess Early-Onset and Genetic Alzheimer’s Disease T2 - ICASSP 2021 - 2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) N2 - The PSEN1-E280A or Paisa mutation is responsible for most of Early-Onset Alzheimer’s (EOA) disease cases in Colombia. It affects a large kindred of over 5000 members that present the same phenotype. The most common symptoms are related to language disorders, where speech fluency is also affected due to the difficulty to access semantic information intentionally. This study proposes the use of acoustic and linguistic methods to extract features from speech recordings and their transcriptions to discriminate people with conditions related to the Paisa mutation. We consider state-of-the-art word-embedding methods like Word2Vec and Bidirectional Encoder Representations from Transformer to process the transcripts. The speech signals are modeled by using traditional acoustic features and speaker embeddings. To the best of our knowledge, this is the first study focused on evaluating genetic Alzheimer’s and EOA using acoustics and linguistics. KW - PSEN1–E280A KW - Alzheimer’s Disease KW - Acoustic Analysis KW - Linguistic Analysis Y1 - 2021 SN - 978-1-7281-7605-5 U6 - https://doi.org/10.1109/ICASSP39728.2021.9414009 SP - 8338 EP - 8342 PB - IEEE ER - TY - CHAP A1 - Scheuerer, Ralph A1 - Haderlein, Tino A1 - Nöth, Elmar A1 - Bocklet, Tobias T1 - Applying X-Vectors on Pathological Speech After Larynx Removal T2 - 2021 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU) N2 - Speaker embeddings extracted from time delayed neural networks (TDNNs) contributed to major recent advancements in speaker recognition and verification. We use an X-Vector system trained on augmented VoxCeleb1 and VoxCeleb2 data to obtain embeddings for pathological speech after total or partial larynx removal. We show that our model is able to effectively distinguish and visualize patient groups when generating embeddings. We further compare various regression models on the task of automatically predicting different perceptual ratings by speech therapists (intelligibility, vocal effort, and overall quality) based on the extracted speaker embeddings. For both patient groups we show Pearson correlations in the range of +0.8; we find that Random Forest and Support Vector Regression produce scores that best resemble the experts' assessments. KW - laryngectomy KW - intelligibility KW - pathological speech KW - x-vectors Y1 - 2021 SN - 978-1-6654-3739-4 U6 - https://doi.org/10.1109/asru51503.2021.9688278 VL - 2021 SP - 1079 EP - 1086 PB - IEEE ER - TY - CHAP A1 - Bundscherer, Maximilian A1 - Schmitt, Thomas H. A1 - Bayerl, Sebastian P. A1 - Auerbach, Thomas A1 - Bocklet, Tobias T1 - An Acoustical Machine Learning Approach to Determine Abrasive Belt Wear of Wide Belt Sanders T2 - 2022 IEEE Sensors N2 - This paper describes a machine learning approach to determine the abrasive belt wear of wide belt sanders used in industrial processes based on acoustic data, regardless of the sanding process-related parameters, Feed speed, Grit Size, and Type of material. Our approach utilizes Decision Tree, Random Forest, k-nearest Neighbors, and Neural network Classifiers to detect the belt wear from Spectrograms, Mel Spectrograms, MFCC, IMFCC, and LFCC, yielding an accuracy of up to 86.1% on five levels of belt wear. A 96% accuracy could be achieved with different Decision Tree Classifiers specialized in different sanding parameter configurations. The classifiers could also determine with an accuracy of 97% if the machine is currently sanding or is idle and with an accuracy of 98.4% and 98.8% detect the sanding parameters Feed speed and Grit Size. We can show that low-dimensional mappings of high-dimensional features can be used to visualize belt wear and sanding parameters meaningfully. KW - Acoustic sensors KW - Abrasive belt wear KW - Tool wear KW - Machine learning KW - Industrial process KW - Wide belt sanding machines Y1 - 2022 SN - 978-1-6654-8464-0 U6 - https://doi.org/10.1109/SENSORS52175.2022.9967324 VL - 2022 PB - IEEE ER - TY - CHAP A1 - Schmitt, Thomas H. A1 - Bundscherer, Maximilian A1 - Drechsel, Ralf A1 - Bocklet, Tobias T1 - Machine learning based optimization of a ceramic bushing manufacturing process T2 - 2022 IEEE Sensors N2 - Machine learning (ML) has shown great promise in a variety of domains in recent years. ML models are known to require large amounts of labeled training data, keeping small to medium-sized business from utilizing them. This paper presents ML based approach to optimize a ceramic bushing manufac-turing process, by predicting the employed press-fit process as a function of press punch position. Accurate predictions would ensure optimal process configuration, guaranteeing quality and reducing waste. Models are trained in a supervised manner to predict the press-fit process and the ceramic defect probabilities as functions of press punch position. We were able to predict the press-fit process with a mean correlation of 0.996 and assess whether the process would damage the ceramic with a mean precision of 96.7%. Our results exemplify how ML can be used to predict and optimize highly specialised processes even with small datasets. KW - Manufacturing KW - machine learning KW - optimization KW - ceramic bushing Y1 - 2022 SN - 978-1-6654-8464-0 U6 - https://doi.org/10.1109/sensors52175.2022.9967124 PB - IEEE ER - TY - CHAP A1 - Klumpp, P. A1 - Bocklet, Tobias A1 - Arias-Vergara, T. A1 - Vásquez-Correa, J. C. A1 - Pérez-Toro, P.A. A1 - Bayerl, Sebastian P. A1 - Orozco-Arroyave, J. R. A1 - Nöth, Elmar T1 - The Phonetic Footprint of Covid-19? T2 - Interspeech 2021 N2 - Against the background of the ongoing pandemic, this year’s Computational Paralinguistics Challenge featured a classification problem to detect Covid-19 from speech recordings. The presented approach is based on a phonetic analysis of speech samples, thus it enabled us not only to discriminate between Covid and non-Covid samples, but also to better understand how the condition influenced an individual’s speech signal. Our deep acoustic model was trained with datasets collected exclusively from healthy speakers. It served as a tool for segmentation and feature extraction on the samples from the challenge dataset. Distinct patterns were found in the embeddings of phonetic classes that have their place of articulation deep inside the vocal tract. We observed profound differences in classification results for development and test splits, similar to the baseline method. We concluded that, based on our phonetic findings, it was safe to assume that our classifier was able to reliably detect a pathological condition located in the respiratory tract. However, we found no evidence to claim that the system was able to discriminate between Covid-19 and other respiratory diseases. KW - COVID-19 Y1 - 2021 U6 - https://doi.org/10.21437/Interspeech.2021-1488 SN - 2958-1796 SP - 441 EP - 445 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Baumann, Ilja A1 - Wagner, Dominik A1 - Bayerl, Sebastian P. A1 - Bocklet, Tobias T1 - Nonwords Pronunciation Classification in Language Development Tests for Preschool Children T2 - Interspeech 2022 N2 - This work aims to automatically evaluate whether the language development of children is age-appropriate. Validated speech and language tests are used for this purpose to test the auditory memory. In this work, the task is to determine whether spoken nonwords have been uttered correctly. We compare different approaches that are motivated to model specific language structures: Low-level features (FFT), speaker embeddings (ECAPA-TDNN), grapheme-motivated embeddings (wav2vec 2.0), and phonetic embeddings in form of senones (ASR acoustic model). Each of the approaches provides input for VGG-like 5-layer CNN classifiers. We also examine the adaptation per nonword. The evaluation of the proposed systems was performed using recordings from different kindergartens of spoken nonwords. ECAPA-TDNN and low-level FFT features do not explicitly model phonetic information; wav2vec2.0 is trained on grapheme labels, our ASR acoustic model features contain (sub-)phonetic information. We found that the more granular the phonetic modeling is, the higher are the achieved recognition rates. The best system trained on ASR acoustic model features with VTLN achieved an accuracy of 89.4% and an area under the ROC (Receiver Operating Characteristic) curve (AUC) of 0.923. This corresponds to an improvement in accuracy of 20.2% and AUC of 0.309 relative compared to the FFT-baseline. Y1 - 2022 U6 - https://doi.org/10.21437/interspeech.2022-10777 SN - 2958-1796 VL - 2022 SP - 3643 EP - 3647 PB - ISCA ER - TY - CHAP A1 - Chen, Wenda A1 - Huang, Jonathan A1 - Bocklet, Tobias T1 - Length- and Noise-Aware Training Techniques for Short-Utterance Speaker Recognition T2 - Interspeech 2020 N2 - Speaker recognition performance has been greatly improved with the emergence of deep learning. Deep neural networks show the capacity to effectively deal with impacts of noise and reverberation, making them attractive to far-field speaker recognition systems. The x-vector framework is a popular choice for generating speaker embeddings in recent literature due to its robust training mechanism and excellent performance in various test sets. In this paper, we start with early work on including invariant representation learning (IRL) to the loss function and modify the approach with centroid alignment (CA) and length variability cost (LVC) techniques to further improve robustness in noisy, far-field applications. This work mainly focuses on improvements for short-duration test utterances (1-8s). We also present improved results on long-duration tasks. In addition, this work discusses a novel self-attention mechanism. On the VOiCES far-field corpus, the combination of the proposed techniques achieves relative improvements of 7.0% for extremely short and 8.2% for full-duration test utterances on equal error rate (EER) over our baseline system. KW - speaker recognition KW - invariant representation learning KW - centroid alignment KW - x-vector KW - far-field Y1 - 2020 U6 - https://doi.org/10.21437/interspeech.2020-2872 SN - 2958-1796 SP - 3835 EP - 3839 PB - ISCA CY - ISCA ER - TY - JOUR A1 - Bayerl, Sebastian P. A1 - Wagner, Dominik A1 - Baumann, Ilja A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - Detecting Vocal Fatigue with Neural Embeddings JF - Journal of Voice N2 - Vocal fatigue refers to the feeling of tiredness and weakness of voice due to extended utilization. This paper investigates the effectiveness of neural embeddings for the detection of vocal fatigue. We compare x-vectors, ECAPA-TDNN, and wav2vec 2.0 embeddings on a corpus of academic spoken English. Low-dimensional mappings of the data reveal that neural embeddings capture information about the change in vocal characteristics of a speaker during prolonged voice usage. We show that vocal fatigue can be reliably predicted using all three types of neural embeddings after 40 minutes of continuous speaking when temporal smoothing and normalization are applied to the extracted embeddings. We employ support vector machines for classification and achieve accuracy scores of 81% using x-vectors, 85% using ECAPA-TDNN embeddings, and 82% using wav2vec 2.0 embeddings as input features. We obtain an accuracy score of 76%, when the trained system is applied to a different speaker and recording environment without any adaptation. KW - Vocal fatigue KW - Neural embeddings KW - Visualization KW - Detection Y1 - 2023 U6 - https://doi.org/10.1016/j.jvoice.2023.01.012 SN - 0892-1997 PB - Elsevier BV ER - TY - CHAP A1 - Bayerl, Sebastian P. A1 - Wagner, Dominik A1 - Nöth, Elmar A1 - Bocklet, Tobias A1 - Riedhammer, Korbinian T1 - The Influence of Dataset Partitioning on Dysfluency Detection Systems T2 - Text, Speech, and Dialogue N2 - This paper empirically investigates the influence of different data splits and splitting strategies on the performance of dysfluency detection systems. For this, we perform experiments using wav2vec 2.0 models with a classification head as well as support vector machines (SVM) in conjunction with the features extracted from the wav2vec 2.0 model to detect dysfluencies. We train and evaluate the systems with different non-speaker-exclusive and speaker-exclusive splits of the Stuttering Events in Podcasts (SEP-28k) dataset to shed some light on the variability of results w.r.t. to the partition method used. Furthermore, we show that the SEP-28k dataset is dominated by only a few speakers, making it difficult to evaluate. To remedy this problem, we created SEP-28k-Extended (SEP-28k-E), containing semi-automatically generated speaker and gender information for the SEP-28k corpus, and suggest different data splits, each useful for evaluating other aspects of methods for dysfluency detection. KW - Computer Science KW - Informatics Y1 - 2022 SN - 9783031162695 U6 - https://doi.org/10.1007/978-3-031-16270-1_35 SN - 0302-9743 SP - 423 EP - 436 PB - Springer International Publishing CY - Cham ER - TY - CHAP A1 - Wagner, Dominik A1 - Bayerl, Sebastian P. A1 - Maruri, Hector A. Cordourier A1 - Bocklet, Tobias T1 - Generative Models for Improved Naturalness, Intelligibility, and Voicing of Whispered Speech T2 - 2022 IEEE Spoken Language Technology Workshop (SLT) N2 - This work adapts two recent architectures of generative models and evaluates their effectiveness for the conversion of whispered speech to normal speech. We incorporate the normal target speech into the training criterion of vector-quantized variational autoencoders (VQ-VAEs) and Mel-GANs, thereby conditioning the systems to recover voiced speech from whispered inputs. Objective and subjective quality measures indicate that both VQ-VAEs and MelGANs can be modified to perform the conversion task. We find that the proposed approaches significantly improve the Mel cepstral distortion (MCD) metric by at least 25% relative to a Disco-GAN baseline. Subjective listening tests suggest that the MelGAN-based system significantly improves naturalness, intelligibility, and voicing compared to the whispered input speech. A novel evaluation measure based on differences between latent speech representations also indicates that our MelGAN-based approach yields improvements relative to the baseline. KW - whispered speech KW - speech conversion KW - VAE KW - GAN KW - generative models Y1 - 2023 SN - 979-8-3503-9690-4 U6 - https://doi.org/10.1109/SLT54892.2023.10022796 SP - 943 EP - 948 PB - IEEE ER - TY - CHAP A1 - Vasquez-Correa, J. C. A1 - Bocklet, Tobias A1 - Orozco-Arroyave, J. R. A1 - Nöth, E. T1 - Comparison of User Models Based on GMM-UBM and I-Vectors for Speech, Handwriting, and Gait Assessment of Parkinson’s Disease Patients T2 - ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) N2 - Parkinson's disease is a neurodegenerative disorder characterized by the presence of different motor impairments. Information from speech, handwriting, and gait signals have been considered to evaluate the neurological state of the patients. On the other hand, user models based on Gaussian mixture models - universal background models (GMMUBM) and i-vectors are considered the state-of-the-art in biometric applications like speaker verification because they are able to model specific speaker traits. This study introduces the use of GMM-UBM and i-vectors to evaluate the neurological state of Parkinson's patients using information from speech, handwriting, and gait. The results show the importance of different feature sets from each type of signal in the assessment of the neurological state of the patients. KW - Parkinson’s disease KW - GMM-UBM KW - ivectors KW - gait analysis KW - handwriting analysis KW - speech analysis Y1 - 2020 SN - 978-1-5090-6631-5 U6 - https://doi.org/10.1109/icassp40776.2020.9054348 SP - 6544 EP - 6548 PB - IEEE ER - TY - JOUR A1 - Schuster, Maria A1 - Arias-Vergara, Tomás A1 - Müller-Hörner, Rainer A1 - Winterholler, Cordula A1 - Bocklet, Tobias T1 - "Verstehen mich mit der Maske eigentlich noch alle?" JF - Geriatrie-Report Y1 - 2020 U6 - https://doi.org/10.1007/s42090-020-0627-3 SN - 1862-5363 VL - 15 IS - 3 SP - 8 EP - 9 PB - Springer Science and Business Media LLC ER - TY - CHAP A1 - Bergler, Matthias A1 - Tolvanen, Juha-Pekka A1 - Zoppelt, Markus A1 - Tavakoli Kolagari, Ramin T1 - Social Engineering Exploits in Automotive Software Security BT - Modeling Human-targeted Attacks with SAM T2 - Proceedings of the 31st European Safety and Reliability Conference (ESREL 2021) N2 - Security cannot be implemented into a system retrospectively without considerable effort, so security must be taken into consideration already at the beginning of the system development. The engineering of automotive software is by no means an exception to this rule. For addressing automotive security, the AUTOSAR and EAST-ADL standards for domain-specific system and component modeling provide the central foundation as a start. The EASTADL extension SAM enables fully integrated security modeling for traditional feature-targeted attacks. Due to the COVID-19 pandemic, the number of cyber-attacks has increased tremendously and of these, about 98 percent are based on social engineering attacks. These social engineering attacks exploit vulnerabilities in human behaviors, rather than vulnerabilities in a system, to inflict damage. And these social engineering attacks also play a relevant but nonetheless regularly neglected role for automotive software. The contribution of this paper is a novel modeling concept for social engineering attacks and their criticality assessment integrated into a general automotive software security modeling approach. This makes it possible to relate social engineering exploits with feature-related attacks. To elevate the practical usage, we implemented an integration of this concept into the established, domain-specific modeling tool MetaEdit+. The tool support enables collaboration between stakeholders, calculates vulnerability scores, and enables the specification of security objectives and measures to eliminate vulnerabilities. KW - automotive systems, social engineering attacks, design, model-based development, modeling, security Y1 - 2021 U6 - https://doi.org/10.3850/978-981-18-2016-8_720-cd SP - 2502 EP - 2509 PB - Research Publishing Services CY - Singapore ER - TY - CHAP A1 - Georges, Munir A1 - Huang, Jonathan A1 - Bocklet, Tobias T1 - Compact Speaker Embedding: lrx-Vector BT - Proceedings of the Annual Conference of the International Speech Communication Association T2 - Interspeech 2020 N2 - Deep neural networks (DNN) have recently been widely used in speaker recognition systems, achieving state-of-the-art performance on various benchmarks. The x-vector architecture is especially popular in this research community, due to its excellent performance and manageable computational complexity. In this paper, we present the lrx-vector system, which is the low-rank factorized version of the x-vector embedding network. The primary objective of this topology is to further reduce the memory requirement of the speaker recognition system. We discuss the deployment of knowledge distillation for training the lrx-vector system and compare against low-rank factorization with SVD. On the VOiCES 2019 far-field corpus we were able to reduce the weights by 28% compared to the full-rank x-vector system while keeping the recognition rate constant (1.83% EER). KW - speaker recognition KW - x-vector KW - low power Y1 - 2020 U6 - https://doi.org/10.21437/interspeech.2020-2106 SN - 2958-1796 SP - 3236 EP - 3240 PB - ISCA CY - ISCA ER - TY - JOUR A1 - Axenie, Cristian A1 - López-Corona, Oliver A1 - Makridis, Michail A. A1 - Akbarzadeh, Meisam A1 - Saveriano, Matteo A1 - Stancu, Alexandru A1 - West, Jeffrey T1 - Antifragility in complex dynamical systems JF - npj Complexity N2 - Antifragility characterizes the benefit of a dynamical system derived from the variability in environmental perturbations. Antifragility carries a precise definition that quantifies a system’s output response to input variability. Systems may respond poorly to perturbations (fragile) or benefit from perturbations (antifragile). In this manuscript, we review a range of applications of antifragility theory in technical systems (e.g., traffic control, robotics) and natural systems (e.g., cancer therapy, antibiotics). While there is a broad overlap in methods used to quantify and apply antifragility across disciplines, there is a need for precisely defining the scales at which antifragility operates. Thus, we provide a brief general introduction to the properties of antifragility in applied systems and review relevant literature for both natural and technical systems’ antifragility. We frame this review within three scales common to technical systems: intrinsic (input–output nonlinearity), inherited (extrinsic environmental signals), and induced (feedback control), with associated counterparts in biological systems: ecological (homogeneous systems), evolutionary (heterogeneous systems), and interventional (control). We use the common noun in designing systems that exhibit antifragile behavior across scales and guide the reader along the spectrum of fragility–adaptiveness–resilience–robustness–antifragility, the principles behind it, and its practical implications. Y1 - 2024 U6 - https://doi.org/10.1038/s44260-024-00014-y SN - 2731-8753 VL - 1 IS - 1 PB - Springer Science and Business Media LLC ER - TY - CHAP A1 - Schuhbauer, Heidi A1 - Brockmann, Patricia A1 - Schötteler, Sebastian T1 - Agile learning: Use of Eduscrum in higher education T2 - Education and New Developments 2024 – Volume 2 N2 - Eduscrum is a methodology that applies agile project management principles to the field of education. The Scrum methodology is an established approach in software development, where iterative product development is carried out by a small development team in work cycles. Originally, Scrum was an agile framework that originated in software development but has since been adapted to various industries, including education. The idea behind Eduscrum is to apply agile project management principles to education, aiming to be more flexible in addressing the needs of learners, fostering collaboration, and enabling continuous feedback. Eduscrum organizes the learning process into sprints, which are time-boxed periods typically lasting one to four weeks. During each sprint, students and teachers work collaboratively to achieve specific learning objectives. Scrum defines roles which are sometimes difficult to adapt to the educational context. The role of the product owner is often taken on by the teacher, who defines the learning goals and prioritizes the content. The Scrum master moderates the learning process, removes obstacles, and ensures that the Scrum framework is followed effectively. Students take on the role of the development team. Regular meetings, often referred to as "stand-ups" or "Scrum meetings,"are held to discuss progress and challenges. These meetings provide a platform for students and teachers to collaborate, share insights, and identify any impediments. The Scrum process includes artifacts, like a backlog to prioritize tasks and charts which track the progress of the sprints. One of the key principles of Eduscrum is flexibility. It allows for adjustments based on student needs, enabling a dynamic and responsive learning environment. Eduscrum promotes regular feedback loops between students and teachers, fostering a culture of continuous improvement. Collaboration is a central theme in Eduscrum. Students work together on projects, share ideas, and contribute to the overall learning experience. The paper describes how the methodology is used in a master’s level class for computer science. Content of the course subject is social network analysis. The students work together in small groups. They perform together a self-chosen project and write a research paper together. Every process step is accompanied by the professor and an assistant, in accordance with Eduscrum. Each project group gets constant feedback. The students are guided step by step through the whole process. In the course evaluation, the students state their high learning outcomes, but they find that the projects are a lot of work. KW - Agile learning, Eduscrum, higher education, science education, social network analysis Y1 - 2024 SN - 78-989-35728-0-1 U6 - https://doi.org/10.36315/2024v2end034 SN - 2184-1489 PB - inScience Press ER - TY - CHAP A1 - Ranzenberger, Thomas A1 - Freier, Carolin A1 - Reinold, Luca A1 - Riedhammer, Korbinian A1 - Schneider, Fabian A1 - Simic, Christopher A1 - Simon, Claudia A1 - Freisinger, Steffen A1 - Georges, Munir A1 - Bocklet, Tobias T1 - A Multidisciplinary Approach to AI-based self-motivated Learning and Teaching with Large Language Models T2 - Proceedings of DELFI 2024 N2 - We present a learning experience platform that uses machine learning methods to support students and lecturers in self-motivated online learning and teaching processes. The platform is being developed as an agile open-source collaborative project supported by multiple universities and partners. The development is guided didactically, reviewed, and scientifically evaluated in several cycles. Transparency, data protection and the copyright compliant use of the system is a central part of the project. The system further employs large language models (LLMs). Due to privacy concerns, we utilize locally hosted LLM instances and explicitly do not rely on available cloud products. Students and lecturers can interact with an LLM-based chatbot in the current prototype. The AI-generated outputs contain cross-references to the current educational video’s context, indicating if sections are based on the lectures context or world knowledge. We present the prototype and results of our qualitative evaluation from the perspective of lecturers and students. KW - Artificial Intelligence in Education; Learning Experience Platform; Open Source Software; Large Language Models Y1 - 2024 U6 - https://doi.org/10.18420/delfi2024_11 SN - 2944-7682 PB - Gesellschaft für Informatik e.V. CY - Bonn ER - TY - JOUR A1 - Brockmann, Patricia A1 - Schuhbauer, Heidi T1 - Generative AI-chatbots in higher education: Challenges and opportunities in student motivation and authentic assessments JF - Education and New Developments N2 - The rapid pace of development in generative artificial intelligence (AI) has resulted in the introduction of extremely advanced chatbots, such as ChatGPT, Google Bard and Copilot. These chatbots are capable of simulating human behavior quite successfully, because they have been trained on huge data sets with large language models (LLM). New avenues for application of generative AI-chatbots have arisen not only in industry, science and medicine, but also in education. The consideration of social and ethical consequences of this new technology has not kept pace with the speed of its development. This paper explores challenges and opportunities which have arisen in higher education with the emergence of generative AI-chatbots. For university students, the temptation to use chatbots to complete their assignments may seem like a good way to save time and effort. The disadvantage is that they may not learn anything by simply copying answers from a chatbot. Inexperienced students may trust the validity of answers which are incorrect, especially if they do not have prerequisite knowledge to evaluate the output of a chatbot. For educators, however, it can be extremely difficult to determine whether students have completed their assignments themselves, or whether they have submitted the output from a chatbot. Instructors have the responsibility to teach university students to avoid plagiarism by judiciously citing the sources used. Furthermore, competencies taught at the university level should go beyond mere reproduction of facts. The development of analytical capabilities and critical thinking often require hard work to learn from mistakes, so-called “productive failure”. Creative teaching methods, such as gamification, may help to motivate students to engage with learning materials. A number of questions which have arisen from these challenges will be addressed in this paper. (1) Can educators determine whether student submissions are original or were written by a chatbot?; (2) Should students be allowed to use generative AI-chatbots to work on assessments? If so, how?; (3) Can creative teaching methods, such as gamification, motivate students to engage in learning?; (4) What kinds of assessments can help to evaluate whether students have achieved learning goals? Challenges encountered in exploring these questions when teaching undergraduate university courses will be presented. Opportunities to increase student engagement and design of authentic assessments will be discussed. KW - Artificial intelligence, chatbots, motivation, gamification, assessment. Y1 - 2024 SN - 978-989-35106-9-8 U6 - https://doi.org/10.36315/2024v2end024 SN - 2184-044X ER - TY - JOUR A1 - Altenbernd, Peter A1 - Gustafsson, Jan A1 - Lisper, Björn A1 - Stappert, Friedhelm T1 - Early execution time-estimation through automatically generated timing models JF - Real-Time Systems N2 - Traditional timing analysis, such as worst-case execution time analysis, is normally applied only in the late stages of embedded system software development, when the hardware is available and the code is compiled and linked. However, preliminary timing estimates are often needed in early stages of system development as an essential prerequisite for the configuration of the hardware setup and dimensioning of the system. During this phase the hardware is often not available, and the code might not be ready to link. This article describes an approach to predict the execution time of software through an early, source-level timing analysis. A timing model for source code is automatically derived from a given combination of hardware architecture and compiler. The model is identified from measured execution times for a set of synthetic training programs, compiled for the hardware platform in question. It can be used to estimate the execution time for code running on the platform: the estimation is then done directly from the source code, without compiling and running it. Our experiments show that, using this model, we can predict the execution times of the final, compiled code surprisingly well. For instance, we achieve an average deviation of 8% for a set of benchmark programs for the ARM7 architecture. KW - Early timing estimates · Timing model identification · WCET analysis Y1 - 2016 U6 - https://doi.org/10.1007/s11241-016-9250-7 SN - 0922-6443 VL - 52 IS - 6 SP - 731 EP - 760 PB - Springer Science and Business Media LLC ER - TY - JOUR A1 - Escobar-Grisales, Daniel A1 - Ríos-Urrego, Cristian-David A1 - Baumann, Ilja A1 - Riedhammer, Korbinian A1 - Nöth, Elmar A1 - Bocklet, Tobias A1 - Garcia, Adolfo A1 - Orozco-Arroyave, Juan rafael T1 - It’s Time to Take Action: Acoustic Modeling of Motor Verbs to Detect Parkinson’s Disease N2 - Pre-trained models generate speech representations that are used in different tasks, including the automatic detection of Parkinson’s disease (PD). Although these models can yield high accuracy, their interpretation is still challenging. This paper used a pre-trained Wav2vec 2.0 model to represent speech frames of 25ms length and perform a frame-by-frame discrimination between PD patients and healthy control (HC) subjects. This fine granularity prediction enabled us to identify specific linguistic segments with high discrimination capability. Speech representations of all produced verbs were compared w.r.t. nouns and the first ones yielded higher accuracies. To gaina deeper understanding of this pattern, representations of motor and non-motor verbs were compared and the first ones yielded better results, with accuracies of around 83% in an independent test set. These findings support well-established neurocognitive models about action-related language highlighted as key drivers of PD. Index Terms: computational paralinguistics, interpretability of pre-trained models, action verbs, Parkinson’s disease KW - computational paralinguistics, interpretability of pre-trained models, action verbs, Parkinson’s disease Y1 - 2024 U6 - https://doi.org/10.21437/Interspeech.2024-2205 ER - TY - CHAP A1 - Brockmann, Patricia T1 - Problem-Based E-Learning to Increase Motivationn of STEM-Students BT - Gamification of a Startup Enterprise T2 - Proceedings of the International Teaching and Education Conference N2 - First semester students often face major challenges in adjusting to university studies. Inexperienced students may not appreciate the practical relevance of abstract, theoretical concepts taught using traditional, instructor-centered lectures. Furthermore, the rise of generative artificial intelligence (GenAI) can mislead some to believe they no longer need to exert effort to learn academic skills. To encourage students to engage in learning material and thus improve retention rates, two complementary teaching methods were integrated into a large, introductory course for first semester STEM students: Problem-Based Learning (PBL) and gamification. Problem-Based Learning was implemented to make the subject matter more meaningful by simulating a real world experience: How to start up a small business. This entrepreneurship task was simulated in an online game, to increase student motivation. Quantitative questionnaires of student motivation and experiences with e-learning were evaluated. Based on input obtained in expert interviews, a fictitious startup enterprise was designed to serve as the used case. An online e-learning game was developed, which leveraged gamification elements to try to increase motivation. Each phase of the startup process was represented as one level of the e-learning game. The e-learning game was tested by a group of first semester students. Their opinions were collected using an anonymous online survey. Aggregated results of the survey are discussed and plans for further research are presented. KW - E-Learning, Gamification, STEM, Problem-Baseed Learning, Used-Case Learning Y1 - 2024 U6 - https://doi.org/10.33422/teduconf.v1i1.746 SN - 3030-1521 VL - 1 IS - 1 PB - Diamond Scientific Publishing ER - TY - CHAP A1 - Marutschke, Daniel Moritz A1 - Brockmann, Patricia T1 - Multicultural Observations in a Three-University Global Distributed Software Development Course BT - A UML Case Study T2 - Proceedings of the 2024 The 6th World Symposium on Software Engineering (WSSE) N2 - In our increasingly interconnected world, software engineering students need increased exposure to not only state of the art technology, but to its usage and development in multicultural environments. This paper reports on findings from a global software engineering course taught at three universities in Japan, Germany, and Indonesia. Questionnaire and interview data gathered ex-ante (before) and ex-post (after) the semester about cultural dimensions and UML usage are analyzed qualitatively and quantitatively. The perceived importance of cultural dimensions was compared to previous two-university classes. Experience reports from instructors are described. Unique dynamics in students’ perception of cultural dimensions could be observed. UML challenges self-reported are consistent with previous research findings. An ex-ante to ex-post comparison supports the implementation of real-world projects taught in PBL-style. These findings together are presented as best practices and lessons learned. KW - Global Software Engineering KW - Education KW - Distributed Teams KW - Cultural Dimenstions KW - UML Y1 - 2024 SN - 979-8-4007-1708-6 U6 - https://doi.org/10.1145/3698062.3698077 VL - 2024 SP - 115 EP - 120 PB - ACM Digital Library ER - TY - CHAP A1 - Bayerl, Sebastian Peter A1 - Wagner, Dominik A1 - Noeth, Elmar A1 - Riedhammer, Korbinian T1 - Detecting Dysfluencies in Stuttering Therapy Using wav2vec 2.0 T2 - Interspeech 2022 N2 - Stuttering is a varied speech disorder that harms an individual’s communication ability. Persons who stutter (PWS) often use speech therapy to cope with their condition. Improving speech recognition systems for people with such non-typical speech or tracking the effectiveness of speech therapy would require systems that can detect dysfluencies while at the same time being able to detect speech techniques acquired in therapy. This paper shows that fine-tuning wav2vec 2.0 [1] for the classification of stuttering on a sizeable English corpus containing stuttered speech, in conjunction with multi-task learning, boosts the effectiveness of the general-purpose wav2vec 2.0 features for detecting stuttering in speech; both within and across languages. We evaluate our method on FluencyBank , [2] and the German therapy-centric Kassel State of Fluency (KSoF) [3] dataset by training Support Vector Machine classifiers using features extracted from the finetuned models for six different stuttering-related event types: blocks, prolongations, sound repetitions, word repetitions, interjections, and – specific to therapy – speech modifications. Using embeddings from the fine-tuned models leads to relative classification performance gains up to 27% w.r.t. F1-score. KW - stuttering KW - pathological speech KW - computational paralinguistics Y1 - 2022 U6 - https://doi.org/10.21437/Interspeech.2022-10908 SP - 2868 EP - 2872 PB - ISCA CY - ISCA ER - TY - CHAP A1 - Seibt, Simon A1 - Chang, Thomas A1 - Latoschik, Marc Erich A1 - Kuth, Bastian A1 - von Rymon Lipinski, Bartosz T1 - Multidimensional image morphing-fast image-based rendering of open 3D and VR environments T2 - Virtual Reality & Intelligent Hardware N2 - In recent years, the demand for interactive photorealistic three-dimensional (3D) environments has increased in various fields, including architecture, engineering, and entertainment. However, achieving a balance between the quality and efficiency of high-performance 3D applications and virtual reality (VR) remains challenging. Methods This study addresses this issue by revisiting and extending view interpolation for image-based rendering (IBR), which enables the exploration of spacious open environments in 3D and VR. Therefore, we introduce multimorphing, a novel rendering method based on the spatial data structure of 2D image patches, called the image graph. Using this approach, novel views can be rendered with up to six degrees of freedom using only a sparse set of views. The rendering process does not require 3D reconstruction of the geometry or per-pixel depth information, and all relevant data for the output are extracted from the local morphing cells of the image graph. The detection of parallax image regions during preprocessing reduces rendering artifacts by extrapolating image patches from adjacent cells in real-time. In addition, a GPU-based solution was presented to resolve exposure inconsistencies within a dataset, enabling seamless transitions of brightness when moving between areas with varying light intensities. Results Experiments on multiple real-world and synthetic scenes demonstrate that the presented method achieves high "VR-compatible" frame rates, even on mid-range and legacy hardware, respectively. While achieving adequate visual quality even for sparse datasets, it outperforms other IBR and current neural rendering approaches. Conclusions Using the correspondence-based decomposition of input images into morphing cells of 2D image patches, multidimensional image morphing provides high-performance novel view generation, supporting open 3D and VR environments. Nevertheless, the handling of morphing artifacts in the parallax image regions remains a topic for future research. KW - Computer graphics; 3D real-time rendering; Computer vision; Image morphing; Virtual reality Y1 - 2024 U6 - https://doi.org/10.1016/j.vrih.2023.06.007 VL - 7 IS - 2 ER -