TY - CHAP A1 - Kousidis, Spyros A1 - Kennington, Casey A1 - Baumann, Timo A1 - Buschmeier, Hendrik A1 - Kopp, Stefan A1 - Schlangen, David T1 - Situationally Aware In-Car Information Presentation Using Incremental Speech Generation: Safer, and More Effective T2 - Proceedings of the EACL 2014 Workshop on Dialogue in Motion, Gothenburg, Sweden N2 - Holding non-co-located conversationswhile driving is dangerous (Horrey and- Wickens, 2006; Strayer et al., 2006), much more so than conversations with physically present, “situated” interlocutors (Drews et al., 2004). In-car dialogue systems typically resemble non-co-located conversations more, and share their negative impact (Strayer et al., 2013). We implemented and tested a simple strategy for making in-car dialogue systems aware of the driving situation, by giving them the capability to interrupt themselves when a dangerous situation is detected,and resume when over. We show that this improves both driving performance and recall of system-presented information, compared to a non-adaptive strategy. KW - in-car information presentation KW - incremental speech generation KW - situation awareness Y1 - 2014 U6 - https://doi.org/10.3115/v1/W14-0212 SP - 68 EP - 72 PB - Association for Computational Linguistics ER - TY - CHAP A1 - Kennington, Casey A1 - Kousidis, Spyros A1 - Baumann, Timo A1 - Buschmeier, Hendrik A1 - Kopp, Stefan A1 - Schlangen, David ED - Miller, Erika ED - Wu, Yuqing T1 - Better Driving and Recall When In-car Information Presentation Uses Situationally-Aware Incremental Speech Output Generation T2 - Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, AutomotiveUI '14, Seattle WA, USA, September 17 - 19, 2014 N2 - It is established that driver distraction is the result of sharing cognitive resources between the primary task (driving) and any other secondary task. In the case of holding conversations, a human passenger who is aware of the driving conditions can choose to interrupt his speech in situations potentially requiring more attention from the driver, but in-car information systems typically do not exhibit such sensitivity. We have designed and tested such a system in a driving simulation environment. Unlike other systems, our system delivers information via speech (calendar entries with scheduled meetings) but is able to react to signals from the environment to interrupt when the driver needs to be fully attentive to the driving task and subsequently resume its delivery. Distraction is measured by a secondary short-term memory task. In both tasks, drivers perform significantly worse when the system does not adapt its speech, while they perform equally well to control conditions (no concurrent task) when the system intelligently interrupts and resumes. Y1 - 2014 UR - 978-1-4503-3212-5 U6 - https://doi.org/10.1145/2667317.2667332 SP - 1 EP - 7 PB - ACM CY - New York, NY ER - TY - CHAP A1 - Kousidis, Spyros A1 - Kennington, Casey A1 - Baumann, Timo A1 - Buschmeier, Hendrik A1 - Kopp, Stefan A1 - Schlangen, David ED - Ali Salah, Albert T1 - A Multimodal In-Car Dialogue System That Tracks The Driver's Attention T2 - Proceedings of the 16th International Conference on Multimodal Interaction, ICMI '14, November 12 - 16 2014, Istanbul N2 - When a passenger speaks to a driver, he or she is co-located with the driver, is generally aware of the situation, and can stop speaking to allow the driver to focus on the driving task. In-car dialogue systems ignore these important aspects, making them more distracting than even cell-phone conversations. We developed and tested a "situationally-aware" dialogue system that can interrupt its speech when a situation which requires more attention from the driver is detected, and can resume when driving conditions return to normal. Furthermore, our system allows driver-controlled resumption of interrupted speech via verbal or visual cues (head nods). Over two experiments, we found that the situationally-aware spoken dialogue system improves driving performance and attention to the speech content, while driver-controlled speech resumption does not hinder performance in either of these two tasks Y1 - 2014 SN - 9781450328852 U6 - https://doi.org/10.1145/2663204.2663244 SP - 26 EP - 33 PB - ACM CY - New York, NY ER - TY - CHAP A1 - Baumann, Timo A1 - Kennington, Casey A1 - Hough, Julian A1 - Schlangen, David T1 - Recognising Conversational Speech: What an Incremental ASR Should Do for a Dialogue System and How to Get There T2 - Proceedings of the International Workshop Series on Spoken Dialogue Systems Technology (IWSDS) 2016 N2 - Automatic speech recognition (ASR) is not only becoming increasingly accurate, but also increasingly adapted for producing timely, incremental output. However, overall accuracy and timeliness alone are insufficient when it comes to interactive dialogue systems which require stability in the output and responsivity to the utterance as it is unfolding. Furthermore, for a dialogue system to deal with phenomena such as disfluencies, to achieve deep understanding of user utterances these should be preserved or marked up for use by downstream components, such as language understanding, rather than be filtered out. Similarly, word timing can be informative for analyzing deictic expressions in a situated environment and should be available for analysis. Here we investigate the overall accuracy and incremental performance of three widely used systems and discuss their suitability for the aforementioned perspectives. From the differing performance along these measures we provide a picture of the requirements for incremental ASR in dialogue systems and describe freely available tools for using and evaluating incremental ASR. Y1 - 2016 ER - TY - CHAP A1 - Baumann, Timo A1 - Schlangen, David ED - Fingscheidt, T. T1 - The INPROTK 2012 release BT - A toolkit for incremental spoken dialogue processing T2 - Sprachkommunikation 2012 : Beiträge zur 10. ITG-Fachtagung vom 26. bis 28. September 2012 in Braunschweig N2 - We describe the 2012 release of INPROTK1, our “Incremental Processing Toolkit“ which combines a powerful and extensible architecture for incremental processing with components for incremental speech recognition and, new to this release, incremental speech synthesis. These components work domainindependently; we also provide example implementations of higher-level components such as natural language understanding and dialogue management that are somewhat more tied to a particular domain. The toolkit is accompanied by evaluation tools for analysing timing behaviour, and we highlight some timing results on conversational speech input in this paper. We offer our toolkit to foster research in this new and exciting area, which promises to help increase the naturalness of behaviours that can be modelled in such systems. KW - Speech synthesis Y1 - 2012 UR - https://ieeexplore.ieee.org/document/6309600 SN - 978-3-8007-3455-9 SP - 147 EP - 150 PB - VDE-Verl CY - Berlin ; Offenbach ER - TY - CHAP A1 - Baumann, Timo A1 - Buß, Okko A1 - Atterer, Michaela A1 - Schlangen, David T1 - Evaluating the Potential Utility of ASR N-Best Lists for Incremental Spoken Dialogue Systems T2 - Proceedings of Interspeech 2009 : speech and intelligence ; 6 - 10 September, 2009, Brighton, UK N2 - The potential of using ASR n-best lists for dialogue systems has often been recognised (if less often realised): it is often the case that even when the top-ranked hypothesis is erroneous, a bet- ter one can be found at a lower rank. In this paper, we describe metrics for evaluating whether the same potential carries over to incremental dialogue systems, where ASR output is consumed and reacted upon while speech is still ongoing. We show that even small N can provide an advantage for semantic process- ing, at a cost of a computational overhead. KW - dialogue systems KW - speech recognition KW - naturallanguage understanding KW - incrementality Y1 - 2009 U6 - https://doi.org/10.21437/Interspeech.2009-318 SP - 1031 EP - 1034 PB - ISCA CY - Brighton, UK ER - TY - CHAP A1 - Baumann, Timo A1 - Atterer, Michaela A1 - Schlangen, David T1 - Assessing and Improving the Performance of Speech Recognition for Incremental Systems T2 - NAACL '09: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, May 31 - June 5, 2009, Boulder, Colorado, USA N2 - In incremental spoken dialogue systems, par- tial hypotheses about what was said are re- quired even while the utterance is still ongo- ing. We define measures for evaluating the quality of incremental ASR components with respect to the relative correctness of the par- tial hypotheses compared to hypotheses that can optimize over the complete input, the tim- ingof hypothesisformationrelative to the por- tion ofthe inputthey areabout, andhypothesis stability, defined as the number of times they are revised. We show that simple incremen- tal post-processing can improve stability dra- matically, at the cost of timeliness (from 90% of edits of hypotheses being spurious down to 10% at a lag of 320ms). The measures are not independent,and we show how system de- signers can find a desired operating point for their ASR. To our knowledge, we are the first to suggest and examine a variety of measures for assessing incremental ASR and improve performance on this basis. Y1 - 2009 UR - https://dl.acm.org/doi/10.5555/1620754.1620810 SN - 978-1-932432-41-1 SP - 380 EP - 388 PB - Association for Computational Linguistics ER - TY - CHAP A1 - Baumann, Timo A1 - Buß, Okko A1 - Schlangen, David T1 - InproTK in Action: Open-Source Software for Building German-Speaking Incremental Spoken Dialogue Systems T2 - Electronic speech signal processing 2010 : proceedings of the 21st conference, Berlin, 8 - 10 September 2010 N2 - We present INPROTK, a toolkit for building incremental spoken dia-logue systems. Incremental spoken dialogue systems (systems that may react whilethe user’s utterance is ongoing) are a fairly recent research topic and allow for ex-citing new features. Even though toolkits exist that help in building conventionaldialogue systems, INPROTK offers both a tested architecture for building incre-mental SDSs as well as many of the building blocks necessary when building suchsystems. With INPROTK a researcher can avoid many of the technical difficulties,which hopefully further fosters research in this area. Y1 - 2010 UR - https://www.researchgate.net/publication/233540470_InproTK_in_Action_Open-Source_Software_for_Building_German-Speaking_Incremental_Spoken_Dialogue_Systems SP - 204 EP - 211 PB - TUDpress CY - Berlin, Germany ER - TY - CHAP A1 - Atterer, Michaela A1 - Baumann, Timo A1 - Schlangen, David T1 - No Sooner Said Than Done: Testing the Incrementality of Semantic Interpretations of Spontaneous Speech T2 - Proceedings of Interspeech 2009 : 6 - 10 September 2009, Brighton, U.K. N2 - Ideally, a spoken dialogue system should react without much delay to a user’s utterance. Such a system would already select an object, for instance, before the user has finished her utterance about moving this particular object to a particular place. A prerequisite for such a prompt reaction is that semantic representations are built up on the fly and passed on to other modules. Few approaches to incremental semantics construction exist, and, to our knowledge, none of those has been systematically tested on a spontaneous speech corpus. In this paper, we develop measures to test empirically on transcribed spontaneous speech to what extent we can create semantic interpretation on the fly with an incremental semantic chunker that builds a frame semantics. KW - incrementality KW - spoken dialogue systems KW - spontaneous speech KW - evaluation Y1 - 2009 U6 - https://doi.org/10.21437/Interspeech.2009-539 SP - 1855 EP - 1858 PB - International Speech Communication Association CY - Brighton, UK ER - TY - CHAP A1 - Baumann, Timo A1 - Schlangen, David T1 - Predicting the Micro-Timing of User Input for an Incremental Spoken Dialogue System that Completes a User’s Ongoing Turn T2 - SIGDIAL 2011 Conference, 12th annual meeting of the Special Interest Group on Discourse and Dialogue, Co-located with ACL HLT 2011, Portland, Oregon, 17 - 18 June 2011 N2 - We present the novel task of predicting tem-poral features of continuations of user input,while that input is still ongoing. We show that the remaining duration of an ongoing word, aswell as the duration of the next can be predicted reasonably well, and we put this information touse in a system that synchronously completesa user’s speech. While we focus on collaborative completions, the techniques presented here may also be useful for the alignment of back-channels and immediate turn-taking in anincremental SDS, or to synchronously monitorthe user’s speech fluency for other reasons. Y1 - 2011 UR - https://www.researchgate.net/publication/220794517_Predicting_the_Micro-Timing_of_User_Input_for_an_Incremental_Spoken_Dialogue_System_that_Completes_a_User%27s_Ongoing_Turn SN - 978-1-61839-242-8 SP - 120 EP - 129 PB - Curran CY - Red Hook, NY ER - TY - CHAP A1 - Buschmeier, Hendrik A1 - Baumann, Timo A1 - Dorsch, Benjamin A1 - Kopp, Stefan A1 - Schlangen, David T1 - Combining Incremental Language Generation and Incremental Speech Synthesis for Adaptive Information Presentation T2 - SIGDIAL '12: Proceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Seoul, South Korea, July 5 - 6, 2012 N2 - Participants in a conversation are normally receptive to their surroundings and their interlocutors, even while they are speaking and can, if necessary, adapt their ongoing utterance. Typical dialogue systems are not receptive and cannot adapt while uttering. We present combin-able components for incremental natural lan-guage generation and incremental speech syn-thesis and demonstrate the flexibility they can achieve with an example system that adapts to a listener's acoustic understanding problems by pausing, repeating and possibly rephrasing problematic parts of an utterance. In an evaluation, this system was rated as significantly more natural than two systems representing the current state of the art that either ignore the interrupting event or just pause; it also has a lower response time. Y1 - 2012 U6 - https://doi.org/10.5555/2392800.2392852 SP - 295 EP - 303 PB - Association for Computational Linguistics ER - TY - CHAP A1 - Heintze, Silvan A1 - Baumann, Timo A1 - Schlangen, David T1 - Comparing Local and Sequential Models for Statistical Incremental Natural Language Understanding T2 - SIGDIAL '10: Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Tokyo, Japan,September 24 - 25, 2010 N2 - Incremental natural language understanding is the task of assigning semantic representations to successively larger prefixes of utterances. We compare two types of statistical models for this task: a) local models, which predict a single class for an input; and b), sequential models, which align a sequence of classes to a sequence of input tokens. We show that, with some modifications, the first type of model can be improved and made to approximate the output of the second, even though the latter is more informative. We show on two different data sets that both types of model achieve comparable performance (significantly better than a baseline), with the first type requiring simpler training data. Results for the first type of model have been reported in the literature; we show that for our kind of data our more sophisticated variant of the model performs better. Y1 - 2010 SN - 978-1-932432-85-5 U6 - https://doi.org/10.5555/1944506.1944508 PB - Association for Computational Linguistics CY - Tokyo, Japan ER - TY - CHAP A1 - Baumann, Timo A1 - Schlangen, David T1 - Open-ended, Extensible System Utterances Are Preferred, Even If They Require Filled Pauses T2 - 14th annual meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL 2013) : Metz, France, 22 - 24 August 2013 N2 - In many environments (e. g. sports commentary), situations incrementally unfold over time and often the future appearance of a relevant event can be predicted, but not in all its details or precise timing. We have built a simulation framework that uses our incremental speech synthesis component to assemble in a timely manner complex commentary utterances. In our evaluation, the resulting output is preferred over that from a baseline system that uses a simpler commenting strategy. Even in cases where the incremental system overcommits temporally and requires a filled pause to wait for the upcoming event, the system is preferred over the baseline. Y1 - 2013 UR - https://aclanthology.org/W13-4042.pdf SP - 280 EP - 283 PB - Curran CY - Red Hook, NY ER - TY - CHAP A1 - Atterer, Michaela A1 - Baumann, Timo A1 - Schlangen, David T1 - Towards Incremental End-of-Utterance Detection in Dialogue Systems T2 - COLING '08: Proceedings of the 22nd International Conference on Computational Linguistics, Manchester United Kingdom August 18 - 22, 2008 N2 - We define the task of incremental or 0-lag utterance segmentation, that is, the task of segmenting an ongoing speech recognition stream into utterance units, and present first results. We use a combination of hidden event language model, features from an incremental parser, and acoustic / prosodic features to train classifiers on real-world conversational data (from the Switchboard corpus). The best classifiers reach an F-score of around 56%, improving over baseline and related work. Y1 - 2008 UR - https://www.researchgate.net/publication/221101778_Towards_Incremental_End-of-Utterance_Detection_in_Dialogue_Systems SN - 978-1-905593-44-6 SP - 11 EP - 14 PB - Association for Computational Linguistics ER - TY - CHAP A1 - Baumann, Timo A1 - Schlangen, David T1 - Inpro_iSS: A Component for Just-In-Time Incremental Speech Synthesis T2 - The 13th Conference of the European Chapter of the Association for Computational Linguistics - proceedings of the System Demonstrations, April 23 - 27 2012, Avignon France N2 - We present a component for incremental speech synthesis (iSS) and a set of applications that demonstrate its capabilities. This component can be used to increase the responsivity and naturalness of spoken interactive systems. While iSS can show its full strength in systems that generate output incrementally, we also discuss how even otherwise unchanged systems may profit from its capabilities. Y1 - 2012 PB - Association for Computational Linguistics (ACL) ER - TY - CHAP A1 - Peldszus, Andreas A1 - Buß, Okko A1 - Baumann, Timo A1 - Schlangen, David T1 - Joint Satisfaction of Syntactic and Pragmatic Constraints Improves Incremental Spoken Language Understanding T2 - EACL 2012 Joint Workshop of LINGVIS & UNCLH, Visualization of Linguistic Patterns and Uncovering Language History from Multilingual Resources, Proceedings of the Workshop, April 23 - 24 2012, Avignon France N2 - We present a model of semantic processing of spoken language that (a) is robust against ill-formed input, such as can be expected from automatic speech recognisers, (b) respects both syntactic and pragmatic constraints in the computation of most likely interpretations, (c) uses a principled, expressive semantic representation formalism (RMRS) with a well-defined model theory, and (d) works continuously (producing meaning representations on a word-by-word basis, rather than only for full utterances) and incrementally (computing only the additional contribution by the new word, rather than re-computing for the whole utterance-so-far). We show that the joint satisfaction of syntactic and pragmatic constraints improves the performance of the NLU component (around 10 % absolute, over a syntax-only baseline). Y1 - 2012 UR - https://www.researchgate.net/publication/233540473_Joint_Satisfaction_of_Syntactic_and_Pragmatic_Constraints_Improves_Incremental_Spoken_Language_Understanding SN - 978-1-937284-19-0 SP - 514 EP - 523 PB - The Association for Computational Linguistics ER - TY - CHAP A1 - Buß, Okko A1 - Baumann, Timo A1 - Schlangen, David T1 - Collaborating on Utterances with a Spoken Dialogue System Using an ISU-based Approach to Incremental Dialogue Management T2 - SIGDIAL 10, Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue, Tokyo Japan September 24 - 25, 2010 N2 - When dialogue systems, through theuse of incremental processing, arenot bounded anymore by strict, non-overlapping turn-taking, a whole range ofadditional interactional devices becomesavailable. We explore the use of one suchdevice, trial intonation. We elaborateour approach to dialogue managementin incremental systems, based on theInformation-State-Update approach, anddiscuss an implementation in a micro-domain that lends itself to the use ofimmediate feedback, trial intonations andexpansions. In an overhearer evaluation,the incremental system was judged as sig-nificantly more human-like and reactivethan a non-incremental version. Y1 - 2010 UR - https://www.researchgate.net/publication/220794407_Collaborating_on_Utterances_with_a_Spoken_Dialogue_System_Using_an_ISU-based_Approach_to_Incremental_Dialogue_Management SN - 978-1-932432-85-5 PB - Association for Computational Linguistics CY - Tokyo, Japan ER - TY - CHAP A1 - Schlangen, David A1 - Baumann, Timo A1 - Atterer, Michaela T1 - Incremental Reference Resolution: The Task, Metrics for Evaluation, and a Bayesian Filtering Model that is Sensitive to Disfluencies T2 - Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, September 11 - 12, 2009, London United Kingdom N2 - In this paper we do two things: a) we discuss in general terms the task of incre mental reference resolution (IRR), in particular resolution of exophoric reference, and specify metrics for measuring the performance of dialogue system components tackling this task, and b) we present a simple Bayesian filtering model of IRR that performs reasonably well just using words directly (no structure information and no hand-coded semantics): it picks the right referent out of 12 for around 50 % of real world dialogue utterances in our test corpus. It is also able to learn to interpret not only words but also hesitations, just as humans have shown to do in similar situations, namely as markers of references tohard-to-describe entities. Y1 - 2009 UR - https://clp.ling.uni-potsdam.de/publications/Schlangen-2009-2.pdf SP - 30 EP - 37 PB - Association for Computational Linguistics CY - London, UK ER - TY - CHAP A1 - Schlangen, David A1 - Baumann, Timo A1 - Buschmeier, Hendrik A1 - Buß, Okko A1 - Kopp, Stefan A1 - Skantze, Gabriel A1 - Yaghoubzadeh, Ramin T1 - Middleware for Incremental Processing in Conversational Agents T2 - Proceedings of the 11th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SigDial 2010), September 24 - 25, 2010, Tokyo Japan N2 - We describe work done at three sites on designing conversational agents capable of incremental processing. We focus on the ‘middleware’ layer in these systems, which takes care of passing around and maintaining incremental information between the modules of such agents. All implementations are based on the abstract model of incremental dialogue processing proposed by Schlangen and Skantze (2009), and the paper shows what different instantiations of the model can look like given specific requirements and application areas. Y1 - 2010 UR - https://www.researchgate.net/publication/47735874_Middleware_for_Incremental_Processing_in_Conversational_Agents SP - 51 EP - 54 PB - Association for Computational Linguistics CY - Tokyo, Japan ER - TY - CHAP A1 - von der Malsburg, Titus A1 - Baumann, Timo A1 - Schlangen, David T1 - TELIDA: A Package for Manipulation and Visualisation of Timed Linguistic Data T2 - Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, September 11 - 12, 2009, London United Kingdom N2 - We present a toolkit for manipulating andvisualising time-aligned linguistic datasuch as dialogue transcripts or languageprocessing data. The package comple-ments existing editing tools by allowingfor conversion between their formats, in-formation extraction from the raw files,and by adding sophisticated, and easily ex-tended methods for visualising the dynam-ics of dialogue processing. To illustratethe versatility of the package, we describeits use in three different projects at our site. Y1 - 2009 U6 - https://doi.org/10.5555/1708376.1708419 SP - 302 EP - 305 PB - Association for Computational Linguistics CY - London, UK ER -