Incremental Dialogue Processing

Spoken language unfolds in time, and is understood and generated in a continuous process: when I speak spontaneously, I don’t plan full sentences which I then merely ‘read out’, and you don’t have to wait for me to finish my utterance before you can start to think about and react to it. This may seem obvious, and yet many branches of linguistics, for different reasons, abstract away from these continuous processes and focus on the sentence as their unit of analysis. In this talk, I will briefly review the evidence for incremental processing, and will then describe a model of such processing that we have recently developed (Schlangen & Skantze, EACL 2009 / Dialogue & Discourse 2011), and two implementations of the model in example dialogue systems (Skantze & Schlangen, EACL 2009; Bus, Baumann & Schlangen, SIGdial 2010), and discuss what we have learned from these implementations.

[1]  Jason D. Williams,et al.  Stability and Accuracy in Incremental Speech Recognition , 2011, SIGDIAL Conference.

[2]  David Schlangen,et al.  Predicting the Micro-Timing of User Input for an Incremental Spoken Dialogue System that Completes a User's Ongoing Turn , 2011, SIGDIAL Conference.

[3]  David Schlangen,et al.  TELIDA: A Package for Manipulation and Visualization of Timed Linguistic Data , 2009, SIGDIAL Conference.

[4]  Lutz Marten,et al.  The Dynamics of Language , 2005 .

[5]  David Schlangen,et al.  From reaction to prediction: experiments with computational models of turn-taking , 2006, INTERSPEECH.

[6]  David Schlangen,et al.  No sooner said than done? testing incrementality of semantic interpretations of spontaneous speech , 2009, INTERSPEECH.

[7]  David Schlangen,et al.  Comparing Local and Sequential Models for Statistical Incremental Natural Language Understanding , 2010, SIGDIAL Conference.

[8]  William D. Marslen-Wilson,et al.  Central processes in speech understanding , 1981 .

[9]  David Schlangen,et al.  Evaluating the potential utility of ASR n-best lists for incremental spoken dialogue systems , 2009, INTERSPEECH.

[10]  David Schlangen,et al.  Incremental Reference Resolution: The Task, Metrics for Evaluation, and a Bayesian Filtering Model that is Sensitive to Disfluencies , 2009, SIGDIAL Conference.

[11]  Gabriel Skantze,et al.  A General, Abstract Model of Incremental Dialogue Processing , 2009, EACL.

[12]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[13]  David Schlangen,et al.  Collaborating on Utterances with a Spoken Dialogue System Using an ISU-based Approach to Incremental Dialogue Management , 2010, SIGDIAL Conference.

[14]  David R. Traum,et al.  Conversational Actions and Discourse Situations , 1997, Comput. Intell..

[15]  David Schlangen,et al.  Evaluation and Optimisation of Incremental Processors , 2011, Dialogue Discourse.

[16]  Ellen Campana,et al.  Incremental understanding in human-computer dialogue and experimental evidence for advantages over nonincremental methods , 2007 .

[17]  R. Levy.  Expectation-based syntactic comprehension , 2008, Cognition.

[18]  Stefan Kopp,et al.  Middleware for Incremental Processing in Conversational Agents , 2010, SIGDIAL Conference.

[19]  David DeVault,et al.  Can I Finish? Learning When to Respond to Incremental Interpretation Results in Interactive Dialogue , 2009, SIGDIAL Conference.

[20]  Jelena Mirkovic,et al.  Incrementality and Prediction in Human Sentence Processing , 2009, Cogn. Sci..

[21]  Massimo Poesio,et al.  Completions, Coordination, and Alignment in Dialogue , 2010, Dialogue Discourse.

[22]  E. Schegloff,et al.  A simplest systematics for the organization of turn-taking for conversation , 1974 .

[23]  WILLIAM MARSLEN-WILSON,et al.  Linguistic Structure and Speech Shadowing at Very Short Latencies , 1973, Nature.

[24]  David Schlangen,et al.  Modelling Sub-Utterance Phenomena in Spoken Dialogue Systems , 2010 .

[25]  G. Altmann,et al.  The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye-movements , 2003 .

[26]  Massimo Poesio,et al.  An Incremental Model of Anaphora and Reference Resolution Based on Resource Situations , 2011, Dialogue Discourse.

[27]  David Schlangen,et al.  Assessing and Improving the Performance of Speech Recognition for Incremental Systems , 2009, NAACL.

[28]  Colin M. Brown,et al.  Anticipating upcoming words in discourse: evidence from ERPs and reading times. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[29]  Herbert H. Clark,et al.  Speaking in time , 2002, Speech Commun..

[30]  Wolfgang Finkler,et al.  Incremental generation for real-time applications , 1995 .