Information und Kommunikation
Refine
Year of publication
Document Type
- conference proceeding (article) (171) (remove)
Is part of the Bibliography
- no (171)
Keywords
- Software (4)
- Shadow IT (3)
- prosody (3)
- speech synthesis (3)
- Business-managed IT (2)
- Causality (2)
- Cloud Computing (2)
- Critical Infrastructure (2)
- IT Sourcing (2)
- Linux (2)
Institute
- Fakultät Informatik und Mathematik (124)
- Fakultät Elektro- und Informationstechnik (46)
- Laboratory for Safe and Secure Systems (LAS3) (22)
- Regensburg Strategic IT Management (ReSITM) (10)
- Labor Industrielle Elektronik (4)
- FuE-Anwenderzentrum Informations- und Kommunikationstechnologien (IKT) (3)
- Labor Optische Übertragungssysteme (3)
- Fakultät Angewandte Natur- und Kulturwissenschaften (2)
- IT-Anwenderzentrum (ITZ) (2)
- Labor Elektroakustik (2)
Begutachtungsstatus
- peer-reviewed (86)
- begutachtet (7)
One of the tasks PAULE[1, 2] solves is finding suitable control parameter (cp-)trajectories for a given target acoustic. These cp-trajectories can be used to synthesize speech with the articulatory speech synthesizer of the VocalTractLab (VTL) [3]. If the target acoustic contains substantial microphone noise or other background noises, occasionally PAULE optimizes not for the speech in the target, but for this background noises. By adding a speech/non-speech classifier to the feedback and planning-loop in PAULE this resynthesis of background noises should be mitigated. Unfortunately, the improvements were minor, which might be due to uninformative gradients of the classifier. The importance of informative gradients and the use classifiers to adapt PAULE to different tasks are explained and discussed.
There is high confidence for the hypothesis that in speech perception the cycles of a θ-oscillation segment the auditory signal into syllables [8]. Yet the functionality of the oscillator generating the θ-oscillation is unknown. We follow the finding that, within an auditory scene, speech is perceived as a stream given by temporal coherence [12]. We work with the hypotheses that the θ-oscillator is driven by temporal features providing this coherence. We propose a new temporal feature called O-distance, which detects the onset of a syllable - the starting point to of a θ-cycle–triggered by the temporal distance from to to the instance of the maximal rise of the loudness curve of the vowel. To extract to from the auditory signal, we use the statistical properties of this distance based on the C-center hypothesis [25], which predicts a close temporal relation of the onset consonants to the onset of a vowel. The statistics are derived from reference O-distance extracted from an articulatory database, where the minima and maxima of the loudness are related to maxima and minima of the lower incisor and tongue tip. To judge the quality of the O-distance extracted from the auditory signal, we regard the temporal deviation of the O-distance to the reference O-distance. Currently we achieve a mean deviation of 34ms.
This paper addresses the challenges and advancements in speech recognition for singing, a domain distinctly different from standard speech recognition. Singing encompasses unique challenges, including extensive pitch variations, diverse vocal styles, and background music interference. We explore key areas such as phoneme recognition, language identification in songs, keyword spotting, and full lyrics transcription. I will describe some of my own experiences when performing research on these tasks just as they were starting to gain traction, but will also show how recent developments in deep learning and large-scale datasets have propelled progress in this field. My goal is to illuminate the complexities of applying speech recognition to singing, evaluate current capabilities, and outline future research directions.
The use of chatbots based on large language models (LLMs) and their impact on society are influencing our learning experience platform Hochschul-Assistenz-System (HAnS). HAnS uses machine learning (ML) methods to support students and lecturers in the online learning and teaching processes [1]. This paper introduces LLM-based features available in HAnS which are using the transcript of our improved Automatic Speech Recognition (ASR) pipeline with an average transcription duration of 45 seconds and an average word error rate (WER) of 6.66% on over 8 hours of audio data of 7 lecture videos. A LLM-based chatbot could be used to answer questions on the lecture content as the ASR transcript is provided as context. The summarization and topic segmentation uses the LLM to improve our learning experience platform. We generate multiple choice questions using the LLM and the ASR transcript as context during playback in a period of 3 minutes and display them in the HAnS frontend.
In this study, we address the complex dynamics of emotional speech and
comprehensively examine the integration of rhythmic and vocal features to recognize emotional patterns. Our exploration is conducted using two German emotional corpora: VMEmo and EmoDB. Employing a combination of supervised methods (here linear discriminant analysis, LDA) and unsupervised techniques (here k-means clustering), we aim to uncover nuanced patterns within the emotional speech in these corpora. The application of LDA highlights salient patterns across different feature sets and focuses on the classification of speakers and prosodic characteristics. In addition, k-means clustering uncovers latent structures that reveal subtle mapping between emotions and speech behavior. Our results suggest that it is possible to cluster data based on prosodic behaviors that are influenced by emotional changes. Although precise mapping to the actual clusters derived from emotional labels could not be fully achieved, the results nonetheless reveal a moderate level of success in this investigation.
This study investigates the effects of speech segmentation methods on speaker recognition models, particularly with regard to the use of rhythmic feature sets. Using three automatic methods and one manual method on the German database of Kiel corpus, segmentation was performed based on the identification of vowel onsets. Subsequently rhythmic variability indices derived from these intervals were calculated and used for principal component analysis and support vector machine model in order to investigate the variation among speakers. The results underline the influence of signal segmentation methods on speaker recognition models.
PROM surveys, used to measure the effect of rehabilitation treatments, are typically filled out on paper, and often suffer from low response rates. Replacing it with a multimodal survey system, supporting touch and speech interaction, could lead to lower hurdles and therefore more data quantity. To do this, it requires task-specific training samples for the Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) to classify spoken answers into one of the standardized PROM answer options.
Due to the lack of training data for medical PROM surveys, we created augmented text samples with each answer option description, combined with different templates. To improve training capabilities, introduce a proper test set, and evaluate the ASR, we also collected 1,797 real voice samples within an empirical study. Further, we incorporate the contextual knowledge of the current question into our NLU architecture by implementing one classifier for every question scale.
Our results reveal that training with empirical data leads to better results than augmented data from templates and original answer option descriptions. Because of participant mislabeling of 33% due to the ambiguity of the task, we receive overall low NLU performances with up to 51.1% accuracy, and rank-1-accuracy up to 79.3%. We also find that our implementation of many scale-specific NLU classifiers significantly outperforms one NLU classifier for all labels, that incorporates the same contextual knowledge after the prediction, by 8 percent points.
Speech Recognition Errors in ASR Engines and Their Impact on Linguistic Analysis in Psychotherapies
(2024)
Modern intervention planning in psychotherapies may benefit from predicting process relevant psychotherapy constructs by automated speech analysis. One essential step is the extraction of relevant linguistic speech markers by ASR engines, which because of highly sensible data, work offline. We analyze transcription errors from NeMo, Whisper, and Wav2Vec2.0, focusing on their impact on linguistic markers usually requiring high quality transcripts. By utilizing part-of-speech tagging, we examine error occurrences among different word types. The Linguistic Inquiry and Word Count (LIWC) software aids in extracting markers. We highlight challenges in transcribing spontaneous speech, prevalent in therapy, and compare results with the Mozilla CommonVoice dataset, which features read speech.
This paper investigated whether predictability-based adjustments in production have listener-oriented consequences in perception. By manipulating the acoustic features of a target syllable in different predictability contexts in German, we tested 40 listeners’ perceptual preference for the manipulation. Four source words underwent acoustic modifications on the target syllable. Our results revealed a general preference for the original (unmodified) version over the modified one. However, listeners generally favored the unmodified version more when the source word had a higher predictable context compared to a less predictable one. The results showed that predictability-based adjustments have perceptual consequences and that listeners have predictability-based expectations in perception.
Recent neural text-to-speech (TTS) models are able to synthesize highly
natural speech signals using deep learning techniques. In practical applications, it can be desirable to have explicit control over the prosody (speech rate, fundamental frequency, and energy) of the synthesized speech. Such controllability can be achieved by adding prosody prediction modules, whose main purpose is to estimate plausible prosody features for each phoneme in the text input. This explicit modeling also allows for changing prosody features at inference time, consequently enabling the adjustment of the prosody in the synthesized audio. In this paper, we evaluate to which extent deliberate manipulation of such prosody features is reflected in the resulting speech audio. We focus particularly on changing the pitch (i.e., fundamental frequency) while applying different normalization strategies.
Generative models for audio are commonly used for music composition, sound effects generation for video game development, audio restoration, voice cloning, etc. The ease of generating indistinguishable fake audio with deep learning poses a major threat to personal privacy, online security, and political discourse. Evaluating the quality and realism of these synthetic utterances is crucial for mitigating the potential for misinformation and harm. To assess this threat, this paper conducts a systematic review, using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), on how these deepfake models are currently evaluated. The analysis of 86 papers shows that the majority of the evaluation is conducted on a machine level and highlights a research gap regarding the human perception of deepfakes. This paper explores various methods and perceptual measures employed in assessing audio deepfakes and evaluating their strengths, limitations, and future directions.
In this work we assess whether there is information in pauses in-between utterances of the same or different speakers that are predictive of the following speaker’s utterance. We present models that connect a person’s visual features before they speak to their upcoming utterance. In our experiments we find that outof-the-box pre-trained models can already reach a better-than-chance performance in correlating video embeddings to utterance embeddings. In contrast, models that attempt to predict the first word after the pause do not outperform a unigram model, indicating that our models do not read lips (based e.g. on co-articulation effects) but rather capture more fundamental aspects of the upcoming utterance.
Wine making is usually considered a domain being far off the processing of speech and language. But in a particular aspect, the domains of speech processing and wine making are related, namely, in the description of wine aromas. These descriptors are used for creating wine expertise as well as more general (advertisement-like) textual representations. In the current paper, we use Natural Language Processing techniques, especially Named Entity Recognition, to identify Aspects and Opinions, reflecting wine characteristics. These are combined with analyses of respective relations (triplet extraction) building Aspect-Opinion-Pairs to establish indicative aroma descriptors, also trying to approach the complex interplay amongst these individual statements. In our experiments, we rely on the Falstaff corpus comprising a huge set of wine descriptions. This results in an average F1 score of around 0.85 for Aspect-Opinion classification. For triplet generation multiple strategies were compared, resulting in an average F1 score of 0.67 in this challenging task. For both tasks we rely only on a handful of manually annotated samples, applying pseudo-labeling methods from seed data to achieve automatic labeling.
Despite the potential of AI, only a small percentage of small and medium-sized enterprises (SMEs) are adopting it due to data issues, expertise gaps, and implementation barriers. Zero-shot learning offers a promising approach for SMEs by minimizing these obstacles. This paper explores the use of zero-shot learning in a real-world NLP classification task on online comments (comparable with intent classification tasks) from the e-learning platform Sofatutor. While finetuning has achieved high accuracy (82.3–86.5%), zero-shot models have shown lower performance (39.3–61.4%) due to different label selection, grouping of different scenarios in one class and the type of classification task. Even if the current accuracy is not sufficient for practical application, pre-filtering the data using zeroshot learning might be a promising option for SMEs.
Wines are complex beverages whose taste can be described either numerically or textually, with the former involving the rating of the intensities of different aroma characteristics often with the help of a wine tasting wheel, and the latter with the help of crisp terms often in a poetic fashion. These are often done with the help of wine sommeliers who with one sniff can describe the wine. Usually, each sommelier has a unique style when it comes to textually describing a wine, research has shown that such differences have no negative impact in correctly classifying wines on the basis of their color, grape variety, region etc. Given the recent advancements in the field of Natural Language Processing, especially with the emergence of Large Language Models, we aim to check the capability of Llama 2 in its ability to generate texts pertaining to a specific color of a wine, given a list of aroma intensities as input prompts. In our experiments, we relied on data from Meininger and Falstaff, and on a combination of domain adaptation and pseudo-labeling techniques to create the corpus to train the Llama 2 model on. Also, we relied on a voting scheme of three differently trained classifiers to evaluate the wine-color specific text generation capabilities of Llama 2. Additionally, we employed the services of domain experts to evaluate the quality of a sample set of texts that was generated by Llama 2.
Our paper introduces a new technology for posture research and training:
the INteractive POsture COrrector, IN-POCO. The device warns its users about unfavorable postures when speaking (e.g., sitting in video conferences) and is thus suitable as an aid for rhetoric trainers. In addition, IN-POCO can also collect time-aligned posture and speech signals for researching prosody-posture relationships in the speech sciences. We outline the motivation for the development of IN-POCO and describe the key technical specifications and operational characteristics. The paper concludes
with a pilot experiment in which we provide initial evidence that, for a communicative (public) speaking task, posture does indeed affect speech prosody in gender-specific ways – in line with claims of rhetoric trainers and guidebooks, and such that an unfavorable (e.g,, humped) posture can be assumed to reduce the speaker's vocal charisma.
Das Continuous-Response-Measurement-Verfahren bildet durch die kontinuierliche Bewertungsmöglichkeit eine wichtige Ergänzung zu den gängigen Methoden im Repertoire der Wirkungsforschung. Um diesen Mehrwert voll ausschöpfen zu können, wird als Verfahrensoptimierung die Entwicklung einer Softwarelösung vorgestellt. Die Überprüfung des optimierten CRM-Verfahrens erfolgt mittels eines Anwendungsfalls aus der sprechwissenschaftlichen Telekommunikationsforschung im Rahmen eines User-Acceptance-Tests. Dabei wird die Funktionalität und Bedienerfreundlichkeit der entwickelten CRM-Softwarelösung unter Beachtung der für die Sprechwirkungsforschung relevanten Kriterien in Form einer A-BStudie getestet.
Das Gesamtergebnis des User-Acceptance-Tests fällt für die Software Evalue positiv aus. Mit Hilfe der Verfahrensoptimierung des CRM-Verfahrens ist eine variabel einsetzbare und damit vielfältig nutzbare CRM-Softwarelösung entstanden.
The growing prevalence of voice assistants has sparked privacy concerns with respect to content privacy and potential human-based attacks such as eavesdropping which make users feel uncomfortable utilizing them in public. To address these challenges, understanding human privacy perceptions in acoustic environments becomes paramount. This understanding can empower voice assistants to accurately quantify privacy perceptions, adapt conversational patterns, and ultimately enhance human-machine interaction. This study draws inspiration from human-tohuman interactions and previous research on acoustic privacy, to quantify privacy perceptions in environments characterized by babble noise. The primary objective is a comprehensive evaluation of both objective and subjective measures to quantitatively capture privacy perceptions in acoustic environments.
Speaker recognition systems often use mel-scaled cepstral coefficients (MFCCs) as main features. In contrast to MFCCs, Godoy et al. (2015) proposed a different type of short-term spectral analysis that provides features related to the lower vocal tract (LVT). They are calculated as the ratio of the acoustic shorttime spectra during the closed and open phases of the glottal oscillation cycles based on a pitch-synchronous analysis. These features were suggested to be particularly speaker-specific and might therefore be suitable to substitute or complement MFCCs in speaker recognition systems. The present study investigated the benefit of these features in an i-vector-based speaker recognition system. Using the LVT features alone, the system achieved a speaker recognition rate of 92.3% with 63 enrolled speakers. When the LVT features were fused with conventional MFCC features, the recognition rate was about equal to the recognition rate using MFCC features alone (> 98%).
Octra Backend ist eine portable web-basierte Infrastruktur für Transkriptionsprojekte, die lokal im Feld oder geschützten Bereichen, im begrenzten Intranet oder weltweit erreichbar im Internet eingesetzt werden kann. Entwicklungsziele waren die Gewährleistung möglichst hoher Sicherheitsanforderungen, eine gute Skalierbarkeit sowie eine einfache Installation auch ohne Administratorrechte. Octra Backend ist in Node.js implementiert und für MacOS, Windows und Linux verfügbar.
Wir repräsentieren eine Bedeutung als Liste von Mustersignalen, und unser Ziel ist es, ein weiteres ankommendes Signal damit zu vergleichen. Die Quantenlogik motiviert die Verwendung von Orthogonalprojektoren, um die gesuchte Ähnlichkeit als Projektionswahrscheinlichkeit darzustellen. Die Ergebnisse des quantenlogischen Verfahrens hängen davon ab, in welcher Weise die Signale vorverarbeitet werden. In diesem Aufsatz untersuchen und diskutieren wir vier verschiedene Möglichkeiten der Vorverarbeitung.
NoiSLU: a Noisy speech corpus for Spoken Language Understanding in the Public Transport Domain
(2024)
The use of local public transport requires the barrier-free purchase of a ticket. Travellers who are not proficient in the local language benefit from a multilingual human(ticket)machine voice interaction. This paper presents a nearly parallel audio dataset with 13218 annotated user queries from 20 speakers for English, German and Dutch. The domain-specific speech corpus can be understood as an evaluation dataset for future research in Spoken Language Understanding (SLU) and thus, it enables researches to improve the quality of human-machine interaction applications. Furthermore, we compare the SLU performance of different compositions of Automatic Speech Recognition (ASR) and Natural Language Understanding (NLU) models in baseline experiments on different test datasets.
Um bei Minimalistischen Grammatiken (MG) Übergenerierung zu vermeiden, kann man Einträge mit leeren Exponenten (ε-Einträge) verwenden. Ein Eintrag besteht aus einem Exponenten, der die Äußerung oder Schrift eines Wortes repräsentiert, einer Merkmalsliste, welche die Syntax kodiert und einem λ-Ausdruck, der die Semantik repräsentiert. Leere Einträge führen allerdings zu einer schlechteren Verwendbarkeit der Grammatik für das Parsen. Die vorliegende Arbeit wird ein Umformungsalgorithmus für MGs vorstellen, sodass die Anzahl der ε-Einträge verringert werden kann, um sie wieder für Parser verwendbar zu machen. Hierzu werden die ε-Einträge mit den anderen Einträgen vorverarbeitet und dadurch neue Einträge geschaffen. Die nun überflüssigen ε-Einträge können dann problemlos entfernt werden. Der Algorithmus wurde anhand von über 260 Zahlwortgrammatiken getestet.
Die vorliegende Studie untersucht, inwiefern sich die in der Phonetik verbreiteten Skripte zur automatisierten Feststellung von verschiedenen Aspekten des Sprechflusses von de Jong et al. zur Beurteilung des Sprachstands bei Kindern eignen und in welcher Art und Weise die Methodik angepasst werden könnte. Dazu wurden Sprachdaten von Vorschulkindern mit Deutsch als Muttersprache bzw. als Zweitsprache mithilfe eines Serious Game zur Sprachstandserhebung elizitiert. Die Audiodaten wurden bezüglich Artikulationsrate, Pausen und Füllpartikeln sowohl automatisiert durch die Skripte als auch manuell annotiert. Die Ergebnisse zeigen, dass sich die Skripte zur Ermittlung der Artikulationsrate mit einer relativ hohen Übereinstimmung mit der manuellen Ermittlung zur Verwendung in Sprachstandserhebungsverfahren eignen. Auch die automatische Erkennung von Sprechpausen weist einen hohen Precision-Wert auf und könnte als Instrument in Sprachstandserhebungen verwendet werden. Eine solche Verwendung würde mit
einer Erweiterung um die hier vorgestellte manuelle Methode zum Annotieren von disfluenten und nicht disfluenten Pausen profitieren. Bei den Füllpartikeln stellte sich die automatisierte Klassifizierung hingegen als weniger geeignet heraus. Hier wurde keine hohe Übereinstimmung mit der menschlichen Annotation festgestellt. Um in der Praxis Verwendung finden zu können, muss diese Methode noch erweitert werden, zum Beispiel durch Miteinbeziehen der Pausenerkennung.
Synchrony of Glottal Area Waveform Parameters During the Production of Obstruents in Vowel Context
(2024)
Obstruents are phonemes which require partial or total obstruction of airflow through the vocal tract. Their articulation also requires adjustments of the laryngeal settings, e. g., an abduction gesture to stop vocal fold vibration for voiceless obstruents. This study investigated the laryngeal settings during the production of voiced and voiceless obstruents in vowel context to analyze the degree of synchrony of the involved glottal gestures. High-speed laryngoscopy images were used to determine the glottal area waveform, from which the time functions of the parameters open quotient (OQ), fundamental frequency (f0), and AC and DC amplitude (ACA and DCA) were calculated and analyzed. Significant correlations were found between all pairs of parameters, with strong correlations between some of them, e.g. Open Quotient and AC Amplitude. Correlations were also either consistently positive or negative for specific pairs of parameters across all investigated phonemes. These results could point to consistent patterns in laryngeal gestures that could enhance articulatory speech synthesis.
Concatenative text-to-speech (TTS) systems remain a widely used cheaper alternative to neural TTS systems. Yet concatenation of prerecorded units entails some drawbacks, such as spectral distortion, the perceptual consequences of which remain unclear. In an attempt to bridge this gap, our study focused on the effect of spectral distortion in vowel formants on perceived speech quality in
naturally-read manipulated German words as well as non-words. More specifically, we explored the distortion effect on a varying number of affected formants, at different magnitude and directionality in two corner vowels /a:/ and /i:/. The results indicate that single formant manipulations have a less pronounced effect on the listeners’ perception compared to multiple formant perturbations. The threshold at which the distortion became generally audible was estimated to lie between 0.4 and 1.0 bandwidth. The directionality of the distortion was not found to be significant.
The continuous advancement of digitization extends beyond educational institutions, giving rise to numerous innovations, particularly in the realm of study information [1]. One avenue for incorporating digital methodologies involves leveraging conversational agents (CAs) [2], serving as interactive interfaces bridging the gap between humans and computers. In the broader context, conversational agents are gaining prominence, offering several benefits to their users. The overarching goal is to comprehensively assist users through these intelligent systems. Consequently, exploring existing university chatbots becomes imperative to discern the areas where they excel. This research aims to scrutinize diverse chatbot systems, delving into their use cases and the challenges they encounter, employing a systematic review. Here it turns out that chatbots support universities the most in the fields of administration, e-learning and mental health. Furthermore, the study will investigate practical experiences on the potential applications and implementation of these systems in university settings, incorporating insights from an online survey and interviews, both made with experts. Here it comes to conclusion that preparation in relation to a chatbot implementation is the key factor to success. Otherwise, a failed system is nearly impossible to be saved, once users lost trust in the system. Therefore, carefully made preparations in the technical and organisational field are necessary to provide a helpful assistant.
Die Therapie-App aphaDIGITAL wird im Rahmen eines Forschungsprojekts entwickelt, um Menschen mit Aphasie im häuslichen Umfeld zu
unterstützen. Das Projekt kombiniert bewährte Therapiemethoden mit digitalen Technologien, einschließlich künstlicher Intelligenz und einem interaktiven Avatar namens Eva. Dieser Artikel betrachtet die Analyse und Entwicklung der Interaktionsmechanismen, die für eine digitale Assistenz von Aphasietherapie den größten Einfluss haben. Es wurden dazu reale Therapiesitzungen zwischen sprachgeschädigten Menschen und ihren Therapeuten konversationsanalytisch nach spezifischen Merkmalen untersucht. Darauf aufbauend wurde durch manuelle Animation ein prototypisches Mundbild modelliert und ein eigenes Artikulationssystem konzipiert, um ein entsprechend authentisches Modell der deutschen Artikulations- und Koartikulationsmuster abzubilden.
This paper describes the usability evaluation of the parts of the CHATU chatbot. The evaluation was conducted with 21 participants. A focus of this paper is the description of the carefully designed evaluation procedure, which aims to avoid textual priming of the participants. The general evaluation procedure can be applied to other speech- or text-based conversational systems, and additional material is provided. The evaluation results show that the usability and user experience of CHATU are positively rated. However, the naturalness and novelty of the interaction are not optimal, and the potential influence of users’ experience with LLMs on the evaluation is discussed.
This paper describes a field study conducted with a museum chatbot at the Städel Museum Frankfurt. The chatbot uses the BERT language model for natural language processing and can be operated via touchscreen as well as via speech input. Prior to the study, hypotheses regarding the user experience of the system were formulated and a system-specific questionnaire was designed, which was used to inquire (among other things) about the perceived quality of the speech output and the frequency of audio guide use in museums. During the interaction with the chatbot, log data was collected and stored in the back-end system. The results show a significant correlation between perceived speech quality and user experience. An exploratory data analysis revealed that participants who used only speech input rated the system as significantly more stimulating than participants who used only touch input. Touch input turned out to be the most efficient input modality in terms of answer correctness and was rated highest regarding pragmatic quality. Interestingly touch input was preferred by younger participants. We discuss our findings and conclude that speech interaction should be seriously considered to create engaging conversational user experiences in museums.
Radio access network based positioning and ranging technologies become more and more important for autonomous driving. Positioning on the 5G Sidelink (SL) is discussed in Release 18 (2022-2024) of the Third Generation Partnership Project. For accurate timing-based ranging, a high temporal and spatial resolution is fundamental. The bandwidth for 5G SL positioning is limited due to sharing the existing SL resources with communication. Therefore, we send the positioning reference signal on two separate Component Carriers, i.e. we use a fragmented spectrum. We introduce a two step algorithm for estimating the time of arrival using the inflection point method. We proved a fragmented spectrum can achieve the same accuracy as a continuous spectrum.
We describe ongoing and proposed work concerning incremental prosody extraction and classification for a spoken dialogue system. The system described will be tightly integrated with the SDS's speech recogntion which also works incrementally. The proposed architecture should allow for more control over the user interaction experience, for example allowing more precise and timely end-of-utterance vs. hesitation distinction, and auditive or visual back-channel generation.
Ideally, a spoken dialogue system should react without much delay to a user’s utterance. Such a system would already select an object, for instance, before the user has finished her utterance about moving this particular object to a particular place. A prerequisite for such a prompt reaction is that semantic representations are built up on the fly and passed on to other modules. Few approaches to incremental semantics construction exist, and, to our knowledge, none of those has been systematically tested on a spontaneous speech corpus. In this paper, we develop measures to test empirically on transcribed spontaneous speech to what extent we can create semantic interpretation on the fly with an incremental semantic chunker that builds a frame semantics.
Affordances manifest possibilities of interaction with a spoken dialogsystem. For example, the act of asking a question affords to the recipient thepossibility of answering. In the system we present, the observable act of maneuveringaffords the possibility of controlling a motion. Our system thus uses the affordanceprinciple to shape the interaction: to trigger the usage of instructions that are easyto understand and process, the system gives immediate visual feedback to afforduser commands that can then be reacted upon. This tightening of the interactionloop requires an incremental processing paradigm to allow fast reactions and to beable to alter ongoing system actions. Our system is a hybrid of incremental andnon-incremental processing components, combining conventional, state graph-basedprocessing, which has the advantage of widely available toolkits and well-understooddialog management, with incremental dialog processing which allows for the tightfeedback loop that provides for quick reactions. We tested our approach in a smalluser study and found that users used simpler and setting-independent commandsmore often and were more efficient when faced with the affordance-based version ofour system.
The potential of using ASR n-best lists for dialogue systems has often been recognised (if less often realised): it is often the case that even when the top-ranked hypothesis is erroneous, a bet- ter one can be found at a lower rank. In this paper, we describe metrics for evaluating whether the same potential carries over to incremental dialogue systems, where ASR output is consumed and reacted upon while speech is still ongoing. We show that even small N can provide an advantage for semantic process- ing, at a cost of a computational overhead.
In incremental spoken dialogue systems, par- tial hypotheses about what was said are re- quired even while the utterance is still ongo- ing. We define measures for evaluating the quality of incremental ASR components with respect to the relative correctness of the par- tial hypotheses compared to hypotheses that can optimize over the complete input, the tim- ingof hypothesisformationrelative to the por- tion ofthe inputthey areabout, andhypothesis stability, defined as the number of times they are revised. We show that simple incremen- tal post-processing can improve stability dra- matically, at the cost of timeliness (from 90% of edits of hypotheses being spurious down to 10% at a lag of 320ms). The measures are not independent,and we show how system de- signers can find a desired operating point for their ASR. To our knowledge, we are the first to suggest and examine a variety of measures for assessing incremental ASR and improve performance on this basis.
We present INPROTK, a toolkit for building incremental spoken dia-logue systems. Incremental spoken dialogue systems (systems that may react whilethe user’s utterance is ongoing) are a fairly recent research topic and allow for ex-citing new features. Even though toolkits exist that help in building conventionaldialogue systems, INPROTK offers both a tested architecture for building incre-mental SDSs as well as many of the building blocks necessary when building suchsystems. With INPROTK a researcher can avoid many of the technical difficulties,which hopefully further fosters research in this area.
We present the novel task of predicting tem-poral features of continuations of user input,while that input is still ongoing. We show that the remaining duration of an ongoing word, aswell as the duration of the next can be predicted reasonably well, and we put this information touse in a system that synchronously completesa user’s speech. While we focus on collaborative completions, the techniques presented here may also be useful for the alignment of back-channels and immediate turn-taking in anincremental SDS, or to synchronously monitorthe user’s speech fluency for other reasons.
We present a component for incremental speech synthesis (iSS) and a set of applications that demonstrate its capabilities. This component can be used to increase the responsivity and naturalness of spoken interactive systems. While iSS can show its full strength in systems that generate output incrementally, we also discuss how even otherwise unchanged systems may profit from its capabilities.
In many environments (e. g. sports commentary), situations incrementally unfold over time and often the future appearance of a relevant event can be predicted, but not in all its details or precise timing. We have built a simulation framework that uses our incremental speech synthesis component to assemble in a timely manner complex commentary utterances. In our evaluation, the resulting output is preferred over that from a baseline system that uses a simpler commenting strategy. Even in cases where the incremental system overcommits temporally and requires a filled pause to wait for the upcoming event, the system is preferred over the baseline.
We present a toolkit for manipulating andvisualising time-aligned linguistic datasuch as dialogue transcripts or languageprocessing data. The package comple-ments existing editing tools by allowingfor conversion between their formats, in-formation extraction from the raw files,and by adding sophisticated, and easily ex-tended methods for visualising the dynam-ics of dialogue processing. To illustratethe versatility of the package, we describeits use in three different projects at our site.
Produkt- bzw. Maschinengeräusche mit deutlich hörbaren tonalen Komponenten werden häufig als lästig empfunden. Wenn zusätzlich eine Interaktion mehrerer tonaler Komponenten in Geräuschen stattfindet, was in realen Szenarien beispielsweise durch die Überlagerung tonaler Komponenten durch verschiedene Aggregate, wie Motoren und Getrieben, stattfindet, dann können Geräusche unterschiedlich tonal, aber auch unterschiedlich konsonant und dissonant wahrgenommen werden. Vorangehende Hörexperimente haben gezeigt, dass die Dissonanz bei ähnlicher Tonhaltigkeit der Geräusche einen großen Einfluss auf die Präferenzurteile hat. Des Weiteren konnte bereits ein grundlegendes Empfindungsmodell für die ”Dissonanz” sowie ein Präferenz-Prädiktor entwickelt werden, die die Subjektivdaten in hoher Güte vorhersagen konnten. In den hier vorgestellten Versuchen wurde das Zusammenspiel zwischen der empfundenen Tonhaltigkeit und der sich aus mehreren Tonkomponenten ergebenden Dissonanz auf die Präferenzurteile untersucht. In der Analyse wird aufgezeigt, auf welche Weise und in welchem Umfang die wahrgenommene Ausgeprägtheit der Tonhaltigkeit durch Interaktion mehrerer Tonkomponenten die allgemeine Geräuschqualität beeinflusst. Da verschiedene Grade der Dissonanz nur durch die gleichzeitige Anwesenheit tonaler Komponenten entstehen können, ist das Zusammenspiel beider Größen bei der Bildung eines Präferenzurteils von einer speziellen Natur. Dies wird sowohl bei der Modellierung der Interaktion beider Größen als auch im Vorhersagemodell der Dissonanz berücksichtigt.
Comparing Local and Sequential Models for Statistical Incremental Natural Language Understanding
(2010)
Incremental natural language understanding is the task of assigning semantic representations to successively larger prefixes of utterances. We compare two types of statistical models for this task: a) local models, which predict a single class for an input; and b), sequential models, which align a sequence of classes to a sequence of input tokens. We show that, with some modifications, the first type of model can be improved and made to approximate the output of the second, even though the latter is more informative. We show on two different data sets that both types of model achieve comparable performance (significantly better than a baseline), with the first type requiring simpler training data. Results for the first type of model have been reported in the literature; we show that for our kind of data our more sophisticated variant of the model performs better.
Participants in a conversation are normally receptive to their surroundings and their interlocutors, even while they are speaking and can, if necessary, adapt their ongoing utterance. Typical dialogue systems are not receptive and cannot adapt while uttering. We present combin-able components for incremental natural lan-guage generation and incremental speech syn-thesis and demonstrate the flexibility they can achieve with an example system that adapts to a listener's acoustic understanding problems by pausing, repeating and possibly rephrasing problematic parts of an utterance. In an evaluation, this system was rated as significantly more natural than two systems representing the current state of the art that either ignore the interrupting event or just pause; it also has a lower response time.
When dialogue systems, through theuse of incremental processing, arenot bounded anymore by strict, non-overlapping turn-taking, a whole range ofadditional interactional devices becomesavailable. We explore the use of one suchdevice, trial intonation. We elaborateour approach to dialogue managementin incremental systems, based on theInformation-State-Update approach, anddiscuss an implementation in a micro-domain that lends itself to the use ofimmediate feedback, trial intonations andexpansions. In an overhearer evaluation,the incremental system was judged as sig-nificantly more human-like and reactivethan a non-incremental version.
We present a model of semantic processing of spoken language that (a) is robust against ill-formed input, such as can be expected from automatic speech recognisers, (b) respects both syntactic and pragmatic constraints in the computation of most likely interpretations, (c) uses a principled, expressive semantic representation formalism (RMRS) with a well-defined model theory, and (d) works continuously (producing meaning representations on a word-by-word basis, rather than only for full utterances) and incrementally (computing only the additional contribution by the new word, rather than re-computing for the whole utterance-so-far). We show that the joint satisfaction of syntactic and pragmatic constraints improves the performance of the NLU component (around 10 % absolute, over a syntax-only baseline).
In this paper we do two things: a) we discuss in general terms the task of incre mental reference resolution (IRR), in particular resolution of exophoric reference, and specify metrics for measuring the performance of dialogue system components tackling this task, and b) we present a simple Bayesian filtering model of IRR that performs reasonably well just using words directly (no structure information and no hand-coded semantics): it picks the right referent out of 12 for around 50 % of real world dialogue utterances in our test corpus. It is also able to learn to interpret not only words but also hesitations, just as humans have shown to do in similar situations, namely as markers of references tohard-to-describe entities.
We describe work done at three sites on designing conversational agents capable of incremental processing. We focus on the middleware layer in these systems, which takes care of passing around and maintaining incremental information between the modules of such agents. All implementations are based on the abstract model of incremental dialogue processing proposed by Schlangen and Skantze (2009), and the paper shows what different instantiations of the model can look like given specific requirements and application areas.
Magnetic garnet films prepared by liquid phase epitaxy on paramagnetic substrates of gadolinium gallium garnet are currently used for the imaging of magnetic field distributions. This application is based on the Faraday rotation which can be strongly enhanced by bismuth incorporation. For gray scale imaging the plane of the sensor film should be an easy plane of magnetization so that no domains nucleate. However, incorporation of bismuth induces a strong positive uniaxial anisotropy perpendicular to the film plane, especially if films of [111] orientation are used. To counteract this unwanted behavior neodymium and/or praseodymium are substituted in addition to bismuth. These two elements cause a very strong negative anisotropy. This is tested experimentally by growing series of garnet films with gradually changing composition and growth parameters.
Optimization of Magnetic Garnet Films for Magneto-Optical Imaging of Magnetic Field Distributions
(2004)
Rare-earth iron garnet films are currently applied for magneto-optical imaging of magnetic field distributions. The physical properties of the films can be controlled by the chemical composition, the growth conditions and the crystallographic orientation. The sensor properties must be optimized according to the application desired. A new optimization method is introduced based on the swing of the photoresponse. An application example is presented. Furthermore, the sensitivity can be strongly enhanced using specific crystallographic orientations which induce an easy plane of magnetization being tilted with respect to the film plane. Experimental results of a [112] oriented garnet film are in good agreement with calculations. The influence of the cubic anisotropy on the sensor performance is discussed. Finally, it is shown that also domain films can be applied for magneto-optical imaging.
This paper describes a case study of assessing student's coding behavior and skills in a realistic development setting. Students had to solve typical programming problems in the context of app development for the Android platform using the Eclipse IDE. Data was analyzed using IDE as well as browser interaction logs. In addition, screen recordings of the students' interaction with the IDE provide further insight. In this paper we present the first results of our ongoing work.
It is simple to query a relational database because all columns of the tables are known and the language SQL is easily applicable. In NoSQL, there usually is no fixed schema and no query language. In this article, we present NotaQL, a data-transformation language for wide-column stores. NotaQL is easy to use and powerful. Many MapReduce algorithms like filtering, grouping, aggregation and even breadth-first-search, PageRank and other graph and text algorithms can be expressed in two or three short lines of code.
The Hierarchical Layer Graph (HL graph) is a promising network topology for wireless networks with variable transmission ranges. It was introduced and analyzed by Meyer auf der Heide et al. 2004. In this paper we present a distributed, localized and resource-efficient algorithm for constructing this graph. The qualtiy of the HL graph depends on the domination radius and the publication radius, which affect the amount of interference in the network. These parameters also determine whether the HL graph is a c-spanner, which implies an energy-efficient topology. We investigate the performance on randomly distributed node sets and show that the restrictions on these parameters derived from a worst case analysis are not so tight using realistic settings. Here, we present the results of our extensive experimental evaluation, measuring congestion, dilation and energy. Congestion includes the load that is induced by interfering edges. We distinguish between congestion and realistic congestion where we also take the signal-to-interference ratio into account. Our experiments show that the HL graph contains energy-efficient paths as well as paths with a few number of hops while preserving a low congestion.
This paper overviews the state of the art in reverberant speech processing from the speech recognition viewpoint. First, it points out that the key to successful reverberant speech recognition is to account for long-term dependencies between reverberant observations obtained from consecutive time frames. Then, a diversity of approaches that exploit the long-term dependencies in various ways is described, ranging from signal and feature dereverberation to acoustic model compensation tailored to reverberation. A framework for classifying those approaches is presented to highlight similarities and differences between them.
A novel framework based on conditional emission densities for
hidden Markov models (HMMs) is proposed in this contribution to integrate speech enhancement systems with automatic speech recognition systems. In the training phase, the observed feature vectors, corrupted by background noise and reverberation, together with estimates for the interference as provided by the speech enhancement system are used for training joint densities of the observations and the interference estimates. In the decoding phase, the joint densities are transformed to conditional densities of the observed features given the interference estimates.
Thus, front end processing can be exploited for obtaining interference estimates, and the estimation errors can be modeled very effectively in a data-driven way. Connected digit recognition experiments in a simulated reverberant environment show the potential of the proposed approach: HMMs with the proposed conditional densities outperform various configurations of conventional HMMs in the logarithmic melspectral domain. This is a first step towards using conditional densities for creating synergies between front end and back end.
Index Terms: speech enhancement, robust speech recognition,
dereverberation, conditional HMM emission densities, frame-
by-frame model adaptation.
Analysen auf NoSQL-Datenbanken sind oft langdauernd und die Ergebnisse fur den Benutzer haufig schwer verst ̈andlich. Wir prasentieren eine Möglichkeit, Datenmengen aus Wide-Column Stores mittels der Transformationssprache NotaQL zu transformieren sowie zu aggregieren und die Ergebnisse in Form von Diagrammen dem Benutzer darzustellen. Dabei kommen Sampling-Techniken zum Einsatz, um die Berechnung auf Kosten der Genauigkeit zu beschleunigen. Das von uns verwendete iterative Samplingverfahren sorgt fur eine kontinuierliche Verbesserung der Berechnungsgenauigkeit und bietet zudem Möglichkeiten zur Genauigkeitsabschätzung, die in Form von Konfidenzintervallen in den Diagrammen dargestellt werden kann.
The rising adoption of NoSQL technology in enterprises causes a heterogeneous landscape of different data stores. Different stores provide distinct advantages and disadvantages, making it necessary for enterprises to facilitate multiple systems for specific purposes. This resulting polyglot persistence is difficult to handle for developers since some data needs to be replicated and aggregated between different and within the same stores. Currently, there are no uniform tools to perform these data transformations since all stores feature different APIs and data models. In this paper, we present the transformation language NotaQL that allows cross-system data transformations. These transformations are output-oriented, meaning that the structure of a transformation script is similar to that of the output. Besides, we provide an aggregation-centric approach, which makes aggregation operations as easy as possible.
Wir pr asentieren SQL Island, ein neuartiges browserbasiertes Lernspiel,welches auf dem Konzept der Text-Adventures basiert. Nach einem Flugzeugabsturzlandet die Spielfigur auf einer Insel. Man redet mit Bewohnern, sammelt Gegenst ̈andeund k ̈apft gegen B ̈osewichte. Die Besonderheit bei diesem Spiel ist jedoch, dass derSpieler seine Figur lediglich mittels SQL-Befehlen steuert. Alle n ̈otigen Befehle wer-den zun ̈achst pr ̈asentiert, sodass keine Vorerfahrung notwendig ist. Nach etwa einerStunde Spielzeit beherrscht der Spieler SELECT-, UPDATE- und DELETE-Anfragensowie Gruppierungen, Aggregationsfunktionen und Joins. Das Spiel kann online aufhttp://www.sql-island.de kostenlos und ohne Registrierung gespielt werden.
While high-level software components simplify the programming of grid applications and Web services increase their interoperability, developing such components and configuring the interconnecting services is a demanding task. In this paper, we consider the combination of Higher-Order Components (HOCs) with the Fractal component model and the ProActive library.
HOCs are parallel programming components, made accessible on the grid via Web services that use a special class loader enabling code mobility: executable code can be uploaded to a HOC, allowing one to customize the HOC. Fractal simplifies the composition of components and the ProActive library offers a generator for automatically creating Web services from components composed with Fractal, as long as all the parameters of these services have primitive types.
Taking all the advantages of HOCs, ProActive and Fractal together, the obvious conclusion is that composing HOCs using Fractal and automatically exposing them as Web services on the grid via ProActive minimizes the required efforts for building complex grid systems. In this context, we solved the problem of exchanging code-carrying parameters in automatically generated Web services by integrating the HOC class loading mechanism into the ProActive library.
Holding non-co-located conversationswhile driving is dangerous (Horrey and- Wickens, 2006; Strayer et al., 2006), much more so than conversations with physically present, “situated” interlocutors
(Drews et al., 2004). In-car dialogue systems typically resemble non-co-located conversations more, and share their negative impact (Strayer et al., 2013). We implemented and tested a simple strategy
for making in-car dialogue systems aware of the driving situation, by giving them the capability to interrupt themselves when a dangerous situation is detected,and resume when over. We show that this improves both driving performance and recall of system-presented information,
compared to a non-adaptive strategy.
When humans speak, they do not plan their full utterance inall detail before beginning to speak, nor do they speak piece-by-piece and ignoring their full message – instead humans usepartial representations in which they fill in the missing partsas the utterance unfolds. Incremental speech synthesizers, incontrast, have not yet made use of partial representations and theinformation contained there-in.We analyze the quality of prosodic parameter assignments(pitch and duration) generated from partial utterance specifi-cations (substituting defaults for missing features) in order todetermine the requirements that symbolic incremental prosodymodelling should meet. We find that broader, higher-level infor-mation helps to improve prosody even if lower-level informationabout the near future is yet unavailable. Furthermore, we findthat symbolic phrase-level or utterance-level information is mosthelpful towards the end of the phrase or utterance, respectively,that is, when this information is becoming available even in theincremental case. Thus, the negative impact of incremental pro-cessing can be minimized by using partial representations thatare filled in incrementally.
Automatic speech recognition (ASR) technology has been developed to such a level that off-the-shelf distributed speech recognition services are available (free of cost), which allow researchers to integrate speech into their applications with little development effort or expert knowledge leading to better results compared with previously used open-source tools.
Often, however, such services do not accept language models or grammars but process free speech from any domain. While results are very good given the enormous size of the search space, results frequently contain out-of-domain words or constructs that cannot be understood by subsequent domain-dependent natural language understanding (NLU) components. We present a versatile post-processing technique based on phonetic distance that integrates domain knowledge with open-domain ASR results, leading to improved ASR performance. Notably, our technique is able to make use of domain restrictions using various degrees of domain knowledge, ranging from pure vocabulary restrictions via grammars or N-Grams to restrictions of the acceptable utterances. We present results for a variety of corpora (mainly from human-robot interaction) where our combined approach significantly outperforms Google ASR as well as a plain open-source ASR solution.
Human speakers plan and deliver their utterances incrementally, piece-by-piece, and it is obvious that their choice regarding phonetic details (and the details' peculiarities) is rarely determined by globally optimal solutions. In contrast, parametric speech synthesizers use a full-utterance context when optimizing vocoding parameters and when determing HMM states. Apart from being cognitively implausible, this impedes incremental use-cases, where the future context is often at least partially unavailable. This paper investigates the `locality' of features in parametric speech synthesis voices and takes some missing steps towards better HMM state selection and prosody modelling for incremental speech synthesis.
It is established that driver distraction is the result of sharing cognitive resources between the primary task (driving) and any other secondary task. In the case of holding conversations, a human passenger who is aware of the driving conditions can choose to interrupt his speech in situations potentially requiring more attention from the driver, but in-car information systems typically do not exhibit such sensitivity. We have designed and tested such a system in a driving simulation environment. Unlike other systems, our system delivers information via speech (calendar entries with scheduled meetings) but is able to react to signals from the environment to interrupt when the driver needs to be fully attentive to the driving task and subsequently resume its delivery. Distraction is measured by a secondary short-term memory task. In both tasks, drivers perform significantly worse when the system does not adapt its speech, while they perform equally well to control conditions (no concurrent task) when the system intelligently interrupts and resumes.
When a passenger speaks to a driver, he or she is co-located with the driver, is generally aware of the situation, and can stop speaking to allow the driver to focus on the driving task. In-car dialogue systems ignore these important aspects, making them more distracting than even cell-phone conversations. We developed and tested a "situationally-aware" dialogue system that can interrupt its speech when a situation which requires more attention from the driver is detected, and can resume when driving conditions return to normal. Furthermore, our system allows driver-controlled resumption of interrupted speech via verbal or visual cues (head nods). Over two experiments, we found that the situationally-aware spoken dialogue system improves driving performance and attention to the speech content, while driver-controlled speech resumption does not hinder performance in either of these two tasks
Robots should appropriately give reasons for their actions
when these actions affect a human’s action or goal space. Communicating reasons may help the human understand the robot’s intents and may initiate joint action, i. e., accepting the robot’s goals and cooperating on the robot’s actions. However, to be efficient, the communication of reasons should be limited to the necessary rather than to completeness, conforming to the Gricean Maxim of Quantity. Furthermore, what is necessary only becomes apparent as the situation evolves and hence, for seamless interaction, ongoing utterances must be adapted as they happen. We present a system that flexibly gives reasons in a reduced setting in which the robot needs to intrude a human’s personal space in order to reach its goal.
We propose to use a model of personal space to initiate communication while passing a human thereby acknowledging that humans are not just a special kind of obstacle to be avoided but potential interaction partners. As a simple form of interaction, our system communicates an apology while closely passing a human. To this end, we present a software architecture that integrates a social-spaces knowledge base and a component for incremental speech production. Incrementality ensures that the robot’s utterance can be adapted to fit the developing situation in a natural way. Observer ratings show that personal-space intrusion is perceived as both natural and polite if the robot has the capability to utter and adapt an apology in an incremental way whereas it is perceived as unfriendly if the robot intrudes personal space without saying anything. Moreover, the robot is perceived as less natural if it does not adapt.
Incremental speech synthesis aims at delivering the synthetic voice while the sentence is still being typed. One of the main challenges is the online estimation of the target prosody from a partial knowledge of the sentence's syntactic structure. In the context of HMM-based speech synthesis, this typically results in missing segmental and suprasegmental features, which describe the linguistic context of each phoneme. This study describes a voice training procedure which integrates explicitly a potential uncertainty on some contextual features. The proposed technique is compared to a baseline approach (previously published), which consists in substituting a missing contextual feature by a default value calculated on the training set. Both techniques were implemented in a HMM-based Text-To-Speech system for French, and compared using objective and perceptual measurements. Experimental results show that the proposed strategy outperforms the baseline technique for this language.
The Spoken Wikipedia project unites volunteer readers of encyclopedic entries. Their recordings make encyclopedic knowledge accessible to persons who are unable to read (out of alexia, visual impairment, or because their sight is currently occupied, e. g. while driving). However, on Wikipedia, recordings are available as raw audio files that can only be consumed linearly, without the possibility for targeted navigation or search. We present a reading application which uses an alignment between the recording, text and article structure and which allows to navigate spoken articles, through a graphical or voice-based user interface (or a combination thereof). We present the results of a usability study in which we compare the two interaction modalities. We find that both types of interaction enable users to navigate articles and to find specific information much more quickly compared to a sequential presentation of the full article. In particular when the VUI is not restricted by speech recognition and understanding issues, this interface is on par with the graphical interface and thus a real option for browsing the Wikipedia without the need for vision or reading.
We present a corpus of time-aligned spoken data of Wikipedia articles as well as the pipeline that allows to generate such corpora for many
languages. There are initiatives to create and sustain spoken Wikipedia versions in many languages and hence the data is freely available,
grows over time, and can be used for automatic corpus creation. Our pipeline automatically downloads and aligns this data. The resulting
German corpus currently totals 293h of audio, of which we align 71h in full sentences and another 86h of sentences with some missing
words. The English corpus consists of 287h, for which we align 27h in full sentence and 157h with some missing words. Results are publically available.
Das ständige Umblättern von Noten ist für Musiker ein wiederkehrendes Problem. Dieses wird häufig durch einen Assistenten des Musikers, dem sogenannten Notenwender, gelöst. Diese Unterstützung haben allerdings viele Musiker nur selten während des Übens. In diesem Artikel stellen wir eine Anwendung für mobile Geräte vor, die auf verschiedene Arten das Umblättern von Klavierpartituren unterstützt. In einer Studie mit professionellen Musikern und Klavierschülern wurden diese Arten gegeneinander abgewogen. Die Ergebnisse zeigen auf, dass computer-unterstütztes Blättern Vorteile gegenüber herkömmlichem Blättern hat.
With the increase of centralization of resources in IT-infrastructure and the growing amount of cloud services, database management systems (DBMS) will be more and more outsourced to Infrastructure-as-a-Service (IaaS) providers. The outsourcing of entire databases, or the computation power for processing Big Data to an external provider also means that the provider has full access to the information contained in the database. In this article we propose a feasible solution with Order-Preserving Encryption (OPE) and further, state of the art, encryption methods to sort and process Big Data on external resources without exposing the unencrypted data to the IaaS provider. We also introduce a proof-of-concept client for Google BigQuery as example IaaS Provider.
Automatic speech recognition (ASR) is not only becoming increasingly
accurate, but also increasingly adapted for producing timely, incremental output. However, overall accuracy and timeliness alone are insufficient when it comes to interactive dialogue systems which require stability in the output and responsivity to the utterance as it is unfolding. Furthermore, for a dialogue system to deal with
phenomena such as disfluencies, to achieve deep understanding of user utterances these should be preserved or marked up for use by downstream components, such as language understanding, rather than be filtered out. Similarly, word timing can be informative for analyzing deictic expressions in a situated environment and should
be available for analysis. Here we investigate the overall accuracy and incremental performance of three widely used systems and discuss their suitability for the aforementioned perspectives. From the differing performance along these measures we provide a picture of the requirements for incremental ASR in dialogue systems and describe freely available tools for using and evaluating incremental ASR.
Most modern and post-modern poems have developed a post-metrical idea of lyrical prosody that employs rhythmical features of everyday language and prose instead of a strict adherence to rhyme and metrical schemes. This development is subsumed under the term free verse prosody. We present our methodology for the large-scale analysis of modern and post-modern poetry in both their written form and as spoken aloud by the author. We employ language processing tools to align text and speech, to generate a null-model of how the poem would be spoken by a naïve reader, and to extract contrastive prosodic features used by the poet. On these, we intend to build our model of free verse prosody, which will help to understand, differentiate and relate the different styles of free verse poetry. We plan to use our processing scheme on large amounts of data to iteratively build models of styles, to validate and guide manual style annotation, to identify further rhythmical categories, and ultimately to broaden our understanding of free verse poetry. In this paper, we report on a proof-of-concept of our methodology using smaller amounts of poems and a limited set of features. We find that our methodology helps to extract differentiating features in the authors’ speech that can be explained by philological insight. Thus, our automatic method helps to guide the literary analysis and this in turn helps to improve our computational models.
Predictive incremental parsing produces syntactic representations of sentences as they are produced, e.g. by typing or speaking. In order to generate connected parses for such unfinished sentences, upcoming word types can be hypothesized and structurally integrated with already realized words. For example, the presence of a determiner as the last word of a sentence prefix may indicate that a noun will appear somewhere in the completion of that sentence, and the determiner can be attached to the predicted noun. We combine the forward-looking parser predictions with backward-looking N-gram histories and analyze in a set of experiments the impact on language models, i.e. stronger discriminative power but also higher data sparsity. Conditioning N-gram models, MaxEnt models or RNN-LMs on parser predictions yields perplexity reductions of about 6%. Our method (a) retains online decoding capabilities and (b) incurs relatively little computational overhead which sets it apart from previous approaches that use syntax for language modeling. Our method is particularly attractive for modular systems that make use of a syntax parser anyway, e.g. as part of an understanding pipeline where predictive parsing improves language modeling at no additional cost.
Engineering is based on the understanding of causes and effects. Thus, causality should also guide the safety assessment of complex systems such as autonomous driving cars. To ensure the safety of the intended functionality of these systems, normative regulations like ISO 21448 recommend scenario-based testing. An important task here is to identify critical scenarios, so-called edge and corner cases. Data-driven approaches to this task (e.g. based on machine learning) cannot adequately address a constantly changing operational design domain. Model-based approaches offer a remedy – they allow including different sources of knowledge (e.g. data, human experts) into safety considerations. With this paper, we outline a novel approach for ensuring automotive system safety. We propose to use structural causal models as a probabilistic modelling language to combine knowledge about an open-context environment from different sources. Based on these models, we investigate parameter configurations that are candidates for critical scenarios. In this paper, we first discuss some aspects of scenario-based testing. We then provide an informal introduction to causal models and relate their development lifecycle to the established V-model. Finally, we outline a generic workflow for using causal models to identify critical scenarios and highlight some challenges that arise in the process.
With the increase in demand of services in the automotive industry, automotive enterprises prefer to collaborate with other qualified cross-domain partners to provide complex automotive functions (or services), such as autonomous driving, OTA (Over The Air) vehicle update, V2X (Vehicle-to-Vehicle communication), etc. One key element in cross-domain enterprise collaboration is the mutual agreement between interfaces of software components. In this context, model-to-model mappings of software component models of heterogeneous frameworks for automotive services and to explore the synergies in their interface semantics, have become an essential factor in improving the interoperability among the automotive and other cross-domain enterprises. However, one of the challenges in achieving cross-domain component interface model-to-model mappings at an application level lies in detecting the interface semantics and the semantic relations that are conveyed in different component models in different frameworks. This paper addresses this challenge using a Model Driven Architecture (MDA) based analytical approach to explore interface semantic synergies in the cross-domain component meta-models that are used for automotive services. The approach applies manual semantic checking measurements at an application interface level to understand the meanings and relations between the different meta-model entities of cross-domain framework software components. In this research, we attempt to ensure that interface description models of software components from heterogeneous frameworks can be compared, correlated and re-used for automotive services based on semantic synergies. We have demonstrated our approach using component meta-models from cross-domain enterprises, that are used for the automotive application domain.
In recent years, mapping of application software components’ ontologies semantically emerged as a big research challenge in automotive application domain that manipulates several cross-enterprise synergy knowledge application frameworks. The same knowledge formalized by different experts in different vehicle application frameworks leads to heterogeneous representations of components’ interface data. Consequently, this causes the most daunting impediment in semantic interoperability between the service components in cooperative automotive systems. From a modeling perspective, in the absence of standardized domain-based unified modeling techniques, the orchestration and resolution of semantic data interoperability between various vehicle application frameworks’ components’ interface models remain a challenge. However, this challenge could be addressed using ontological metamodeling by specifying semantic associations between components’ interface model concepts based on the domain knowledge. Apart from the semantic mapping of interface ontological metamodels, this work also defines quality metrics to determine the degree of semantic alignment achieved between the various interface ontologies. Additionally, to reduce development time and cost towards semantic interoperability, this work proposes a semi-automated plugin tool for the applicability of the evaluated quality metrics to semantic mapping of real-world components’ interface models.
Development and verification of modern, dependable automotive systems require appropriate modelling approaches. Classic automotive safety is described by the normative regulations ISO 26262, its relative ISO/PAS 21448, and their respective methodologies. In recent publications, an emerging demand to combine environmental influences, machine learning, or reasoning under uncertainty with standard-compliant analysis techniques can be noticed. Therefore, adapting established methods like FTA and proper tool support is necessary. We argue that Bayesian Networks (BNs) can be used as a central component to address and merge these demands. In this paper, we present our Open-Source Python package BayesianSafety. First, we review how BNs relate to data-driven methods, model-to-model transformations, and causal reasoning. Together with FTA and ETA, these models form the core functionality of our software. After describing currently implemented features and possibilities of combining individual modelling approaches, we provide an informal view of the tool’s architecture and of the resulting software ecosystem. By comparing selected publicly available safety and reliability analysis libraries, we outline that many relevant methodologies yield specialized implementations. Finally, we show that there is a demand for a flexible, unifying analysis tool that allows researching system safety by using multi-model and multi-domain approaches.
With autonomous driving, the system complexity of vehicles will increase drastically. This requires new approaches to ensure system safety. Looking at standards like ISO 26262 or ISO/PAS 21448 and their suggested methodologies, an increasing trend in the recent literature can be noticed to incorporate uncertainty. Often this is done by using Bayesian Networks as a framework to enable probabilistic reasoning. These models can also be used to represent causal relationships. Many publications claim to model cause-effect relations, yet rarely give a formal introduction of the implications and resulting possibilities such an approach may have. This paper aims to link the domains of causal reasoning and automotive system safety by investigating relations between causal models and approaches like FMEA, FTA, or GSN. First, the famous “Ladder of Causation” and its implications on causality are reviewed. Next, we give an informal overview of common hazard and reliability analysis techniques and associate them with probabilistic models. Finally, we analyse a mixed-model methodology called Hybrid Causal Logic, extend its idea, and build the concept of a causal shell model of automotive system safety.
Supervisory Control and Data Acquisition (SCADA) systems are used to control and monitor components within the energy grid, playing a significant role in the stability of the system. As a part of critical infrastructures, components in these systems have to fulfill a variety of different requirements regarding their dependability and must also undergo strict audit procedures in order to comply with all relevant standards. This results in a slow adoption of new functionalities. Due to the emerged threat of cyberattacks against critical infrastructures, extensive security measures are needed within these systems to protect them from adversaries and ensure a stable operation. In this work, a solution is proposed to integrate extensive security measures into current systems. By deploying additional security-gateways into the communication path between two nodes, security features can be integrated transparently for the existing components. The developed security-gateway is compliant to all regulatory requirements and features an internal architecture based on the separation-of-concerns principle to increase its security and longevity. The viability of the proposed solution has been verified in different scenarios, consisting of realistic field tests, security penetration tests and various performance evaluations.
Today's cyberphysical systems are increasingly prone to misuse. To secure existing and future software systems, introducing concepts of IT-Security and Secure Software Engineering (SecSE) in Software Engineering (SE) courses is essential for academic education of future software engineers. This is not only important for computer science students, but also for engineering students studying topics of computing and SE. However, only little research exists on integrating these topics into traditional SE courses, especially for engineering students in non-computer science majors. To narrow this gap, this paper contributes with the design and evaluation of an exercise on modeling misuse cases alongside use cases, based on the inductive teaching method problem-based learning (PBL). The exercise is part of an educational design research investigating which learning content and teaching methods are suitable for integrating IT-Security and SecSE topics into traditional SE education of engineering students to convey factual knowledge as well as raise awareness and interest for both topics during software development. We present the integration of the exercise design into a traditional SE course for engineering students and its evaluation to examine its suitability. We evaluated the exercise design regarding the suitability of the design components, the learning content of misuse cases and the intended learning goals as well as its impact on students' motivation, and their interest in IT-security. The paper then presents indications on the feasibility and success of the exercise design for teaching misuse cases to engineering students and sparking their interest in IT-Security.
In the real-time systems sector, various task models and corresponding tests exist to model and verify the schedulability of task sets on the system at hand. While those models and schedulability tests have intensively been studied from a theoretical point of view, it is hard to ma e use of them to compare the actual execution behavior of scheduling algorithms on a real system. In contrast to schedulability tests, simulators can help to investigate the performance of specific scheduling algorithms. One of the most generalized task models to describe parallel tasks is the Directed Acyclic Graph model that allows to represent tasks as a series of subtasks that depict the potentially parallel computations and precedence constraints that denote the order in which the subtasks are allowed to execute.
In this paper, we investigate various scheduling algorithms for the Directed Acyclic Graph model. For that, we first recapitulate the examined scheduling algorithms in detail and point out relevant differences. Subsequently, we present the evaluation of different global and federated scheduling algorithms using fine-grained parallel tasks. To this end, we generate random Directed Acyclic Graph tasks and simulate their execution on multiprocessor systems using scheduling algorithms such as global rate-monotonic and semi-federated scheduling as well as global scheduling policies using the thread pool model.
Networked control systems as e.g., battery management systems, smart grids or vehicular systems, consist of sensors, actuators and controllers with a communication network in the control loop. The data rate and the reliability of the underlying communication network are key factors since delays or message losses directly affect the system control. In addition, the processor load caused by the communication is significant as it influences the calculation of system states and the setting of control parameters. The power consumption of the communication network has a further impact on the energy efficiency of the respective application. In this paper, the communication technologies Controller Area Network (CAN), Controller Area Network Flexible Data-rate (CAN FD) and Ethernet are compared in the context of networked control systems with focus on a decentralized battery management system. First, the message processing time and the processor load are measured. With regard to energy efficiency, the maximum power consumption is determined. The Bit Error Rates (BER) and the Residual Error Rates (RER) are calculated to evaluate the reliability. Finally, the receive FIFO load under high traffic conditions is examined. Index Terms—Networked control systems, decentralized battery management system, microcontrollers, communication systems, Ethernet, Controller Area Network (CAN), Controller Area Network Flexible Data-rate (CAN FD) energy efficiency, energy consumption, bit error rate, residual error rate, processor load.
Modern cyber-physical systems, such as autonomous vehicles, advanced driver assistance systems, automation systems and battery management systems, result in extended communication requirements regarding the reliability and the availability. The Controller Area Network (CAN) is a broadcast-based protocol which is still used as a standard for serial communication between individual microcontrollers due to its reliability and low power consumption. In addition, it provides mechanisms for detecting transmission errors and retransmitting messages in the event of an error. The enhancement CAN Flexible Data-Rate (CAN FD) offers increased data rates and transmission rates in order to meet the data throughput requirements. In this paper, the mechanisms for reliable data transmission in a CAN FD network are analyzed. To improve reliability, a second identical CAN-FD network is added to the system, using the additional CAN interface already available on common microcontrollers. The redundant communication network is examined in terms of failure rates and the mean time to failure. The reliability over the operation time is calculated for the single and the redundant version of the CAN FD network using the failure rate limits of the ASIL levels.
We present a comprehensive analysis of the neural audio-visual synchrony evaluation tool SyncNet. We assess the agreement of SyncNet scores vis-a-vis human perception and whether we can use these as a reliable metric for evaluating audio-visual lip-synchrony in generation tasks with no ground truth reference audio-video pair. We further look into the underlying elements in audio and video which vitally affect synchrony using interpretable explanations from SyncNet predictions and analyse its susceptibility by introducing adversarial noise. SyncNet has been used in numerous papers on visually-grounded text-to-speech for scenarios such as dubbing. We focus on this scenario which features many local asynchronies (something that SyncNet isn’t made for).
Over the last three decades, the Controller Area Network (CAN) has become the dominant communication in embedded systems. Especially for automotive systems it offers advantages including high robustness, low error rate and high reliability combined with low power consumption. Therefore, learning the basics of this bus system is substantial in this field. Nowadays, various media about the functionality and use of CAN exist which make it easy to read into the topic. But often, theory alone is not sufficient. To deepen the understanding, practical implementation contributes significantly. However, affordable and easy-to-use CAN devices for training purposes are scarce. Existing equipment can be divided into expensive professional devices, which have many functions and inexpensive ones for hobbyists, which require difficult configurations. Therefore, a practical solution is a low-budget device equipped with an overlay which deals with the time consuming configurations. This paper covers the development of a python interface for a purchasable cost effective CAN device for Windows OS. The intention is to create an easy-to-use program that enables beginners to get in touch with CAN and collect practical experience. At the start, a brief explanation of the CAN functionality is given. After that, we introduce the hardware used in this project. Next, the software part covers the development of the interface and the integration of this interface into python-can. Furthermore, a virtual playground is introduced for testing purposes. Also, to demonstrate the functionality of the interface, a test program is executed in conjunction with a logic analyzer.
This article compares the standard electrical method of partial discharge detection with a novel optical detection method based on silicon photomultipliers. A third, complementary, single-loop antenna method is added to represent the ultra-high frequency method commonly used in gas-insulated switchgear/lines. A trio of air-insulated electrode designs that simulate the fundamental fault/discharge types in gaseous insulation (protrusion – corona discharge, floating conductive particle, surface discharges) are employed. Phase-resolved partial discharge activity patterns are compiled for each electrode design. The patterns are analyzed using spatial statistics and the interpretation of the obtained data trends explained by means of an example. Ultimately, the consistency and reliability of discharge detection by the optical methods for each fault/discharge type are evaluated, and suggestions for improvement are made.
The INPROTK 2012 release
(2012)
We describe the 2012 release of INPROTK1, our “Incremental Processing Toolkit“ which combines a powerful and extensible architecture for incremental processing with components for incremental speech recognition and, new to this release, incremental speech synthesis. These components work domainindependently; we also provide example implementations of higher-level components such as natural language understanding and dialogue management that are somewhat more tied to a particular domain. The toolkit is accompanied by evaluation tools for analysing timing behaviour, and we highlight some timing results on conversational speech input in this paper. We offer our toolkit to foster research in this new and exciting area, which promises to help increase the naturalness of behaviours that can be modelled in such systems.
This paper applies design science research methodology to iteratively develop a framework for measuring and communicating IT business value from a CIO perspective. The framework design is based on analysis and integration of literature combined with empirical findings. The framework was evaluated by CIO interviews and a practical feasibility study. The results show that IT business value can be measured and communicated using our framework by applying six consecutive process steps. Thereby, IT business value is not a single number but a set of quantitative and qualitative metrics relevant to stakeholders. Our framework represents a novel and integrated approach on how CIOs can select appropriate metrics, measure, and communicate IT business value to stakeholders. In addition, the paper provides insights for CIOs on how to be successful in IT business value management.
This paper presents ongoing work in incremental speech synthesis that enables a system to adapt speech delivery to unforeseen changes in the timing of motor events (e. g. a robot actuator working faster or slower than anticipated) in order to improve the coordination of speech and gestures for deictic expressions.
In simultaneous interpreting, human experts incrementally construct and extend partial hypotheses about the source speaker’s message, and start to verbalize a corresponding message in the target language, based on a partial translation – which may have to be corrected occasionally. They commence the target utterance in the hope that they will be able to finish understanding the source speaker’s message and determine its translation in time for the unfolding delivery. Of course, both incremental understanding and translation by humans can be garden-pathed, although experts are able to optimize their delivery so as to balance the goals of minimal latency, translation quality and high speech fluency with few corrections. We investigate the temporal properties of both translation input and output to evaluate the tradeoff between low latency and translation quality. In addition, we estimate the improvements that can be gained with a tempo-elastic
speech synthesizer.
The relation of syntax and prosody (the syntax-prosody interface) has been an active area of research, mostly in linguistics and typically studied under controlled conditions. More recently, prosody has also been successfully used in the data-based training of syntax parsers. However, there is a gap between the controlled and detailed study of the individual effects between syntax and prosody and the large-scale application of prosody in syntactic parsing with only a shallow analysis of the respective influences. In this paper, we close the gap by investigating the significance of correlations of prosodic realization with specific syntactic functions using linear mixed effects models in a very large corpus of read-out German encyclopedic texts. Using this corpus, we are able to analyze prosodic structuring performed by a diverse set of speakers while they try to optimize factual content delivery. After normalization by speaker, we obtain significant effects, e.g. confirming that the subject function, as compared to the object function, has a positive effect on pitch and duration of a word, but a negative effect on loudness.
After overcoming the traditional metrics, modern and postmodern poetry developed a large variety of ‘free verse prosodies’ that falls along a spectrum from a more fluent to a more disfluent and choppy style. We present a method, grounded in philological analysis and theories on cognitive (dis)fluency, to analyze this ‘free verse spectrum’ into six classes of poetic styles as well as to differentiate three types of poems with enjambments. We use a model for automatic prosodic analysis of spoken free verse poetry which uses deep hierarchical attention networks to integrate the source text and audio and predict the assigned class. We then analyze and fine-tune the model with a particular focus on enjambments and in two ways: we drill down on classification performance by analyzing whether the model focuses on similar traits of poems as humans would, specifically, whether it internally builds a notion of enjambment. We find that our model is similarly good as humans in finding enjambments; however, when we employ the model for classifying enjambment-dominated poem types, it does not pay particular attention to those lines. Adding enjambment labels to the training only marginally improves performance, indicating that all other lines are similarly informative for the model.
We show how to classify the phrasing of readout poems with the help of machine learning algorithms that use manually engineered features or automatically learn representations. We investigate modern and postmodern poems from the webpage lyrikline, and focus on two exemplary rhythmical patterns in order to detect the rhythmic phrasing: The Parlando and the Variable Foot. These rhythmical patterns have been compared by using two important theoretical works: The Generative Theory of Tonal Music and the Rhythmic Phrasing in English Verse. Using both, we focus on a combination of four different features: The grouping structure, the metrical structure, the time-span-variation, and the prolongation in order to detect the rhythmic phrasing in the two rhythmical types. We use manually engineered features based on text-speech alignment and parsing for classification. We also train a neural network to learn its own representation based on text, speech and audio during pauses. The neural network outperforms manual feature engineering, reaching an f-measure of 0.85.
One of the most important patterns in ancient as well as modern poetry is the enjambment, the continuation of a sentence beyond the end of a line, couplet, or stanza. The paper reports first activities towards the development of a digital tool to analyze the accentuation of poetic enjambments in readout poetry. The aim in this contribution is to recognize two forms of enjambment (emphasized and unemphasized) in poems using audio and text data. We use data from lyrikline which is a major online portal for spoken poetry whereas poems are read aloud by the original authors. We identified by hermeneutical means based on literary analysis a total of 69 poems being characteristic for the use of enjambments in modern and postmodern German poetry and train classifiers to differentiate the emphasized/unemphasized ategorization. A remarkable result of our automated analyses (and to our knowledge the first data-driven analysis of this kind) is the identification of a cultural difference in the accentuation of enjambments: statistically speaking, poets from the former GDR tend to emphasize the enjambment, whereas poets from the FRG do not. We use features derived from speech-to-text alignment and statistical parsing information such as pause lengths, number of lines with verbs, and number of lines with punctuation. The best classification results, calculated by the F-measure, for the both types of enjambment (emphasized/unemphasized) is 0.69.
We present the open-source extensible dialog manager DialogOS.
DialogOS features simple finite-state based dialog management
(which can be expanded to more complex DM strategies via a full-fledged scripting language) in combination with integrated speech recognition and synthesis in multiple languages.
DialogOS runs on all major platforms, provides a simple-to-use
graphical interface and can easily be extended via well-defined
plugin and client interfaces, or can be integrated server-side into
larger existing software infrastructures. We hope that DialogOS
will help foster research and teaching given that it lowers the bar
of entry into building and testing spoken dialog systems and provides paths to extend one’s system as development progresses.