The Representation of Multimodal User Interface Dialogues Using Discourse Pegs

The three-tiered discourse representation defined in (Luperfoy, 1991) is applied to multimodal human-computer interface (HCI) dialogues. In the applied system the three tiers are (1) a linguistic analysis (morphological, syntactic, sentential semantic) of input and output communicative events including keyboard-entered command language atoms, NL strings, mouse clicks, output text strings, and output graphical events; (2) a discourse model representation containing one discourse object, called a peg, for each construct (each guise of an individual) under discussion; and (3) the knowledge base (KB) representation of the computer agent's 'belief' system which is used to support its interpretation procedures. I present evidence to justify the added complexity of this three-tiered system over standard two-tiered representations, based on (A) cognitive processes that must be supported for any non-idealized dialogue environment (e.g., the agents can discuss constructs not present in their current belief systems), including information decay, and the need for a distinction between understanding a discourse and believing the information content of a discourse; (B) linguistic phenomena, in particular, context-dependent NPs, which can be partially or totally anaphoric; and (C) observed requirements of three implemented HCI dialogue systems that have employed this three-tiered discourse representation.

[1]  Damaris M. Ayuso.  Discourse Entities in Janus , 1989, ACL.

[2]  Lauri Karttunen,et al.  Discourse Referents , 1969, COLING.

[3]  James M. Peters,et al.  A Knowledge-Based , 1988 .

[4]  Philip R. Cohen,et al.  Synergistic use of direct manipulation and natural language , 1989, CHI '89.

[5]  Fred Landman.  Pegs and Alecs , 1986, TARK.

[6]  James D. Hollan,et al.  An introduction to HITS: Human Interface Tool Suite , 1991 .

[7]  Wolfgang Wahlster,et al.  User and discourse models for multimodal communication , 1991 .

[8]  Greg Carlson,et al.  A unified analysis of the English bare plural , 1977 .

[9]  Jeroen Groenendijk,et al.  Formal methods in the study of language. part 2 , 1981 .

[10]  Jeannette G. Neal,et al.  Multi-Modal References in Human-Computer Dialogue , 1988, AAAI.

[11]  Barbara J. Grosz,et al.  The Representation and Use of Focus in a System for Understanding Dialogs , 1977, IJCAI.

[12]  Lauri Karttunen.  WHAT MAKES DEFINITE NOUN PHRASES DEFINITE , 1968 .

[13]  Deborah A. Dahl,et al.  Reference resolution in PUNDIT , 1990 .

[14]  Inderjeet Mani,et al.  Knowledge and natural language processing , 1990, CACM.

[15]  Jeroen Groenendijk,et al.  Formal methods in the study of language : [proceedings of the 3rd Amsterdam colloquium, 25-28.03.1980], part 2 , 1981 .

[16]  Sharon L. Oviatt,et al.  Spoken language in interpreted telephone dialogues , 1990, ICSLP.

[17]  Susann Luperfoy.  Discourse pegs: a computational analysis of context-dependent referring expressions , 1992 .

[18]  Elaine Rich,et al.  An Architecture for Anaphora Resolution , 1988, ANLP.

[19]  Ramanathan V. Guha,et al.  CYC: A Midterm Report , 1990, AI Mag..

[20]  Candace L. Sidner,et al.  Discourse Structure and the Proper Treatment of Interruptions , 1985, IJCAI.

[21]  H. H. Clark,et al.  Collaborating on contributions to conversations , 1987 .

[22]  Irene Heim,et al.  The semantics of definite and indefinite noun phrases : a dissertation , 1982 .