Modelling incremental self-repair processing in dialogue

Self-repairs, where speakers repeat themselves, reformul ate or restart what they are saying, are pervasive in human dialogue. These phenomena provide a wind o into real-time human language processing. For explanatory adequacy, a model of dial gue must include mechanisms that account for them. Artificial dialogue agents also need this c apability for more natural interaction with human users. This thesis investigates the structu e of self-repair and its function in the incremental construction of meaning in interaction. A corpus study shows how the range of self-repairs seen in dia logue cannot be accounted for by looking at surface form alone. More particularly it analy ses a string-alignment approach and shows how it is insufficient, provides requirements for a sui table model of incremental context and an ontology of self-repair function. An information-theoretic model is developed which address es these issues along with a system that automatically detects self-repairs and edit terms on transcripts incrementally with minimal latency, achieving state-of-the-art results. Additi onally it is shown to have practical use in the psychiatric domain. The thesis goes on to present a dialogue model to interpret an d ge erate repaired utterances incrementally. When processing repaired rather than fluent tterances, it achieves the same degree of incremental interpretation and incremental repr esentation. Practical implementation methods are presented for an existing dialogue system. Finally, a more pragmatically oriented approach is present ed to model self-repairs in a psycholinguistically plausible way. This is achieved through extending the dialogue model to include a probabilistic semantic framework to perform incremental inference in a reference resolution domain. The thesis concludes that at least as fine-grained a model of c ontext as word-by-word is required for realistic models of self-repair, and context mus t include linguistic action sequences and information update effects. The way dialogue participa nts process self-repairs to make inferences in real time, rather than filter out their disfluency eff ects, has been modelled formally and in practical systems. Submitted for the degree of Doctor of Philosophy Queen Mary University of London

[1]  H. Grice Logic and conversation , 1975 .

[2]  David Schlangen,et al.  Collaborating on Utterances with a Spoken Dialogue System Using an ISU-based Approach to Incremental Dialogue Management , 2010, SIGDIAL Conference.

[3]  Stuart M. Shieber,et al.  A Uniform Architecture for Parsing and Generation , 1988, COLING.

[4]  James F. Allen,et al.  The TRAINS 93 Dialogues , 1995 .

[5]  Robin Cooper,et al.  Records and Record Types in Semantic Theory , 2005, J. Log. Comput..

[6]  Emiel Krahmer,et al.  Computational Generation of Referring Expressions: A Survey , 2012, CL.

[7]  Yang Liu,et al.  Disfluency Detection Using Multi-step Stacked Learning , 2013, NAACL.

[8]  T Florian Jaeger,et al.  On language 'utility': processing complexity and communicative efficiency. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[9]  Frank Keller,et al.  The Entropy Rate Principle as a Predictor of Processing Effort: An Evaluation against Eye-tracking Data , 2004, EMNLP.

[10]  Dov M. Gabbay,et al.  Dynamic syntax - the flow of language understanding , 2000 .

[11]  James H. Martin,et al.  Speech and language processing: an introduction to natural language processing, computational linguistics, and speech recognition, 2nd Edition , 2000, Prentice Hall series in artificial intelligence.

[12]  Noam Chomsky,et al.  वाक्यविन्यास का सैद्धान्तिक पक्ष = Aspects of the theory of syntax , 1965 .

[13]  Stuart M. Shieber,et al.  The Problem of Logical-form Equivalence 2 Review of Natural-language Generation I Have Beneeted Greatly from Discussions With , 1992 .

[14]  Gerard Kempen,et al.  A Framework for Incremental Syntactic Tree Formation , 1987, IJCAI.

[15]  松本 正人 留学生見聞録--コロラドボウルダー校(University of Colorado at Boulder) , 2006 .

[16]  Douglas E. Appelt,et al.  Bidirectional Grammars and the Design of Natural Language Generation Systems , 1987, TINLAP.

[17]  Kevin H. Knuth,et al.  Lattice duality: The origin of probability and entropy , 2013, Neurocomputing.

[18]  Paul Piwek Incremental Conceptualization for Language Production Markus Guhe (University of Edinburgh) Mahwah, NJ: Lawrence Erlbaum Associates (distributed by Psychology Press), 2007, xii+260 pp; hardbound, ISBN 978-0-8058-5624-8, $75.00 , 2008, Computational Linguistics.

[19]  Arash Eshghi,et al.  Uncommon ground : the distribution of dialogue contexts , 2009 .

[20]  Bowen Zhou,et al.  A phrase-level machine translation approach for disfluency detection using weighted finite state transducers , 2006, INTERSPEECH.

[21]  Koenraad De Smedt,et al.  Segment Grammar: a Formalism for Incremental Sentence Generation , 1991 .

[22]  G. Kempen,et al.  A dual system for producing self-repairs in spontaneous speech: Evidence from experimentally elicited corrections , 1987, Cognitive Psychology.

[23]  David DeVault,et al.  Learning to Interpret Utterances Using Dialogue History , 2009, EACL.

[24]  Janet M. Baker,et al.  The Design for the Wall Street Journal-based CSR Corpus , 1992, HLT.

[25]  Arash Eshghi,et al.  Incremental Semantic Construction in a Dialogue System , 2011, IWCS.

[26]  Noam Chomsky,et al.  Lectures on Government and Binding , 1981 .

[27]  Mark Johnson,et al.  How relevant is linguistics to computational linguistics , 2011 .

[28]  M. Pickering,et al.  Toward a mechanistic psychology of dialogue , 2004, Behavioral and Brain Sciences.

[29]  James F. Allen,et al.  Deep Semantic Analysis of Text , 2008, STEP.

[30]  E. Gregoromichelaki,et al.  Splitting the “ I ”s and Crossing the “ You ”s: Context, Speech Acts and Grammar , 2010 .

[31]  Matthew Purver,et al.  Dialogue and compound contributions , 2014, Natural Language Generation in Interactive Systems.

[32]  Arash Eshghi,et al.  Conversational Interactions: Capturing Dialogue Dynamics , 2012 .

[33]  Matthew Purver,et al.  Incremental Generation by Incremental Parsing: Tactical Generation in Dynamic Syntax , 2003, ENLG@EACL.

[34]  Patrick G. T. Healey,et al.  The Distribution of Repair in Dialogue , 2011, CogSci.

[35]  M ShieberStuart,et al.  The problem of logical-form equivalence , 1993 .

[36]  Koenraad De Smedt,et al.  REVISIONS DURING GENERATION USING NON-DESTRUCTIVE UNIFICATION , 1991 .

[37]  Stuart M. Shieber,et al.  Synchronous Tree-Adjoining Grammars , 1990, COLING.

[38]  Alexander Clark,et al.  Measuring Gradience in Speakers' Grammaticality Judgements , 2014, CogSci.

[39]  H. H. Clark,et al.  Using uh and um in spontaneous speaking , 2002, Cognition.

[40]  David D. McDonald Natural Language Generation , 2010, Handbook of Natural Language Processing.

[41]  Jonathan Ginzburg,et al.  The interactive stance : meaning for conversation , 2012 .

[42]  David Schlangen,et al.  Situated Incremental Natural Language Understanding using a Multimodal, Linguistically-driven Update Model , 2014, COLING.

[43]  Mary P. Harper,et al.  Structural event detection for rich transcription of speech , 2004 .

[44]  Frank Keller,et al.  A Psycholinguistically Motivated Version of TAG , 2008, TAG.

[45]  Josef van Genabith,et al.  Finding Common Ground: Towards a Surface Realisation Shared Task , 2010, INLG.

[46]  Günter Neumann,et al.  Interleaving Natural Language Parsing and Generation Through Uniform Processing , 1998, Artif. Intell..

[47]  R. Montague Formal philosophy; selected papers of Richard Montague , 1974 .

[48]  R. Fernández,et al.  Rethinking Overspecification in Terms of Incremental Processing , 2013 .

[49]  Matthew Purver,et al.  Grammar resources for modelling dialogue dynamically , 2009, Cognitive Neurodynamics.

[50]  Ron Artstein,et al.  People hesitate more, talk less to virtual interviewers than to human interviewers , 2013 .

[51]  W. Levelt,et al.  Speaking: From Intention to Articulation , 1990 .

[52]  David Schlangen,et al.  Introduction to the Special Issue on Incremental Processing in Dialogue , 2011 .

[53]  David Schlangen,et al.  Situated incremental natural language understanding using Markov Logic Networks , 2014, Comput. Speech Lang..

[54]  David Milward Incremental Interpretation of Categorial Grammar , 1995, EACL.

[55]  David DeVault,et al.  Incremental interpretation and prediction of utterance meaning for interactive dialogue , 2011, Dialogue Discourse.

[56]  H. Kamp A Theory of Truth and Semantic Representation , 2008 .

[57]  Alexander Clark,et al.  Statistical Representation of Grammaticality Judgements: the Limits of N-Gram Models , 2013, CMCL.

[58]  S. Young,et al.  Scaling POMDPs for Spoken Dialog Management , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[59]  Stefan Kopp,et al.  Situationally Aware In-Car Information Presentation Using Incremental Speech Generation: Safer, and More Effective , 2014, DM@EACL.

[60]  Massimo Poesio,et al.  Completions, Coordination, and Alignment in Dialogue , 2010, Dialogue Discourse.

[61]  Brian Roark,et al.  Probabilistic Top-Down Parsing and Language Modeling , 2001, CL.

[62]  Mary P. Harper,et al.  PCFGs with Syntactic and Prosodic Indicators of Speech Repairs , 2006, ACL.

[63]  Günter Neumann,et al.  A uniform computational model for natural language parsing and generation , 1994 .

[64]  Lenhart K. Schubert,et al.  A Syntactic Framework for Speech Repairs and Other Disruptions , 1999, ACL.

[65]  Elizabeth Shriberg DISFLUENCIES IN SWITCHBOARD , 1996 .

[66]  Matthew Purver,et al.  Incremental Generation by Incremental Parsing , 2003 .

[67]  Okko Buß,et al.  DIUM – An Incremental Dialogue Manager That Can Produce Self-Corrections , 2011 .

[68]  Donald Hindle,et al.  Deterministic Parsing of Syntactic Non-fluencies , 1983, ACL.

[69]  Julian Hough Incremental Semantics Driven Natural Language Generation with Self-Repairing Capability , 2011, RANLP Student Research Workshop.

[70]  Jeroen Groenendijk,et al.  On the semantics of questions and the pragmatics of answers , 1984 .

[71]  Siobhan Chapman Logic and Conversation , 2005 .

[72]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[73]  William Schuler,et al.  A Syntactic Time-Series Model for Parsing Fluent and Disfluent Speech , 2008, COLING.

[74]  Staffan Larsson,et al.  Modelling Language, Action, and Perception in Type Theory with Records , 2012, CSLP.

[75]  Ronnie Cann,et al.  Natural-Language Syntax as Procedures for Interpretation: The Dynamics of Ellipsis Construal , 2011, PRELUDE Project.

[76]  David Schlangen,et al.  What we can learn from Dialogue Systems that don't work: On Dialogue Systems as Cognitive Models , 2009 .

[77]  David McKelvie,et al.  The Syntax of Disfluency in Spontaneous Spoken Language , 1998 .

[78]  Martin Kay,et al.  The MIND System , 1970 .

[79]  Mark Johnson,et al.  Detecting Speech Repairs Incrementally Using a Noisy Channel Approach , 2010, COLING.

[80]  E. Schegloff Repair After Next Turn: The Last Structurally Provided Defense of Intersubjectivity in Conversation , 1992, American Journal of Sociology.

[81]  Philip Thomas,et al.  Self-repair in dialogues of schizophrenics: Effects of hallucinations and negative symptoms , 1992, Brain and Language.

[82]  Gabriel Skantze,et al.  Towards Incremental Speech Generation in Dialogue Systems , 2010, SIGDIAL Conference.

[83]  S. Brennan,et al.  How Listeners Compensate for Disfluencies in Spontaneous Speech , 2001 .

[84]  Michael C. Frank,et al.  Predicting Pragmatic Reasoning in Language Games , 2012, Science.

[85]  Brian Roark,et al.  Deriving lexical and syntactic expectation-based measures for psycholinguistic modeling via incremental top-down parsing , 2009, EMNLP.

[86]  Günter Neumann,et al.  A Head-Driven Approach to Incremental and Parallel Generation of Syntactic Structures , 1990, COLING.

[87]  Staffan Larsson,et al.  A Probabilistic Rich Type Theory for Semantic Interpretation , 2014, EACL 2014.

[88]  Alex Lascarides,et al.  Logics of Conversation , 2005, Studies in natural language processing.

[89]  Andreas Stolcke,et al.  Can Prosody Aid the Automatic Classification of Dialog Acts in Conversational Speech? , 1998, Language and speech.

[90]  Mark Johnson,et al.  The impact of language models and loss functions on repair disfluency detection , 2011, ACL.

[91]  Frank Schilder,et al.  Incremental Generation of Self-corrections Using Underspecification , 2001, CLIN.

[92]  Martin Kay,et al.  Chart Generation , 1996, ACL.

[93]  Mark Steedman,et al.  The NXT-format Switchboard Corpus: a rich resource for investigating the syntax, semantics, pragmatics and prosody of dialogue , 2010, Lang. Resour. Evaluation.

[94]  Staffan Larsson,et al.  Issue-based Dialogue Management , 2002 .

[95]  Michael Strube,et al.  Part-of-Speech Tagging of Transcribed Speech , 2006, LREC.

[96]  C. L. Hamblin QUESTIONS IN MONTAGUE ENGLISH , 1976 .

[97]  Kallirroi Georgila Using Integer Linear Programming for Detecting Speech Disfluencies , 2009, HLT-NAACL.

[98]  Michael J. Fischer,et al.  The String-to-String Correction Problem , 1974, JACM.

[99]  Kevin H. Knuth,et al.  Valuations on Lattices and their Application to Information Theory , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[100]  Stefan Kopp,et al.  Combining Incremental Language Generation and Incremental Speech Synthesis for Adaptive Information Presentation , 2012, SIGDIAL Conference.

[101]  S. B. Needleman,et al.  A general method applicable to the search for similarities in the amino acid sequence of two proteins. , 1970, Journal of molecular biology.

[102]  John J. Godfrey,et al.  SWITCHBOARD: telephone speech corpus for research and development , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[103]  Eric Brill,et al.  An Improved Error Model for Noisy Channel Spelling Correction , 2000, ACL.

[104]  Kepa Joseba Rodríguez,et al.  Form, Intonation and Function of Clarification Requests in German Task-Oriented Spoken Dialogues , 2004 .

[105]  Matthew Purver,et al.  Helping, I Mean Assessing Psychiatric Communication: An Application of Incremental Self-Repair Detection , 2014 .

[106]  Kees van Deemter,et al.  Logical Form Equivalence: the Case of Referring Expressions Generation , 2001, EWNLG@ACL.

[107]  D. Sperber,et al.  Relevance: Communication and Cognition , 1997 .

[108]  Matthew Purver,et al.  Probabilistic Type Theory for Incremental Dialogue Processing , 2014, EACL 2014.

[109]  Eugene Charniak,et al.  Edit Detection and Parsing for Transcribed Speech , 2001, NAACL.

[110]  Gabriel Skantze,et al.  A General, Abstract Model of Incremental Dialogue Processing , 2009, EACL.

[111]  Matthew Purver,et al.  Modelling Expectation in the Self-Repair Processing of Annotat-, um, Listeners , 2013 .

[112]  Paul Hudak,et al.  A theory of incremental computation and its application , 1991, POPL '91.

[113]  Matthew Purver,et al.  On Incrementality in Dialogue: Evidence from Compound Contributions , 2011, Dialogue Discourse.

[114]  Timo Baumann,et al.  Incremental spoken dialogue processing: architecture and lower-level components , 2013 .

[115]  James F. Allen,et al.  Incremental Dialogue System Faster than and Preferred to its Nonincremental Counterpart , 2007 .

[116]  David R. Traum,et al.  Conversational Actions and Discourse Situations , 1997, Comput. Intell..

[117]  R. Cooper Type Theory and Semantics in Flux , 2012 .

[118]  David Schlangen,et al.  Evaluation and Optimisation of Incremental Processors , 2011, Dialogue Discourse.

[119]  Ronnie Cann,et al.  Grammars as Parsers: Meeting the Dialogue Challenge , 2006 .

[120]  P. Healey,et al.  Shared understanding in psychiatrist-patient communication: association with treatment adherence in schizophrenia. , 2013, Patient education and counseling.

[121]  Ehud Reiter,et al.  Book Reviews: Building Natural Language Generation Systems , 2000, CL.

[122]  Matthew Purver,et al.  Helping the medicine go down : Repair and adherence in patient-clinician dialogues , 2012 .

[123]  David Schlangen,et al.  Joint Satisfaction of Syntactic and Pragmatic Constraints Improves Incremental Spoken Language Understanding , 2012, EACL.

[124]  Raquel. FernaÌndez Rovira Non-sentential utterances in dialogue : classification, resolution and use , 2006 .

[125]  Jonas Kuhn,et al.  Combining Referring Expression Generation and Surface Realization: A Corpus-Based Investigation of Architectures , 2013, ACL.

[126]  E. Schegloff,et al.  The preference for self-correction in the organization of repair in conversation , 1977 .

[127]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[128]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[129]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[130]  Pedro M. Domingos MetaCost: a general method for making classifiers cost-sensitive , 1999, KDD '99.

[131]  Hermann Ney,et al.  Improved backing-off for M-gram language modeling , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[132]  E. Schegloff Third Turn Repair , 1997 .

[133]  Matthew Purver,et al.  Strongly Incremental Repair Detection , 2014, EMNLP.

[134]  Andreas Stolcke,et al.  How far do speakers back up in repairs? a quantitatve model , 1998, ICSLP.

[135]  James F. Allen,et al.  Speech repains, intonational phrases, and discourse markers: modeling speakers’ utterances in spoken dialogue , 1999, CL.

[136]  Arash Eshghi,et al.  Incremental Grammar Induction from Child-Directed Dialogue Utterances , 2013, CMCL.

[137]  Koenraad De Smedt,et al.  IPF: an incremental parallel formulator , 1990 .

[138]  Eugene Charniak,et al.  A TAG-based noisy-channel model of speech repairs , 2004, ACL.

[139]  Gerard Kempen,et al.  An Incremental Procedural Grammar for Sentence Formulation , 1987, Cogn. Sci..