MOG 2007: Workshop on Multimodal Output Generation: CTIT Proceedings

This volume brings together presents a wide variety of work offering different perspectives on multimodal generation. Two different strands of work can be distinguished: half of the gathered papers present current work on embodied conversational agents (ECA’s), while the other half presents current work on multimedia applications. Two general research questions are shared by all: what output modalities are most suitable in which situation, and how should different output modalities be combined?

[1]  Willem J. M. Levelt,et al.  Gesture and the communicative intention of the speaker , 2005 .

[2]  Justin Buchler.  The Philosophy of Peirce: Selected Writings , 1941 .

[3]  Ehud Reiter,et al.  Has a Consensus NL Generation Architecture Appeared, and is it Psycholinguistically Plausible? , 1994, INLG.

[4]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[5]  Nicole Chovil.  Social determinants of facial displays , 1991 .

[6]  Maria L. Flecha-Garcia.  Eyebrow Raising, Discourse Structure, and Utterance Function in Face-to-face Dialogue , 2006 .

[7]  Willem J. M. Levelt,et al.  Producing spoken language: A blueprint of the speaker , 1999 .

[8]  Irwin Altman,et al.  Manwatching: A field guide to human behavior. , 1978 .

[9]  Priti Shah,et al.  A Model of the Perceptual and Conceptual Processes in Graph Comprehension , 1998 .

[10]  C. Y. Thielman,et al.  Natural Language with Integrated Deictic and Graphic Gestures , 1989, HLT.

[11]  B. Velichkovsky.  Communicating attention: Gaze position transfer in cooperative problem solving , 1995 .

[12]  Jens Riegelsberger,et al.  Do people trust their eyes more than ears?: media bias in detecting cues of expertise , 2005, CHI EA '05.

[13]  A. Kendon.  Gesture: Visible Action as Utterance , 2004 .

[14]  H. H. Clark,et al.  On the Course of Answering Questions , 1993 .

[15]  Guy Lapalme,et al.  PostGraphe: A System for the Generation of Statistical Graphics and Text , 1996, INLG.

[16]  Mark Steedman.  Cognitive Algorithms: Questions of Representation and Computation in Building a Theory , 1998 .

[17]  Max M. Louwerse,et al.  Focusing Attention with Deictic Gestures and Linguistic Expressions , 2005 .

[18]  Ben Shneiderman,et al.  Speech-Activated Versus Mouse-Activated Commands for Word Processing Applications: An Empirical Evaluation , 1992 .

[19]  A. Kendon.  Movement coordination in social interaction: some examples described. , 1970, Acta psychologica.

[20]  A. Glenberg,et al.  Comprehension of illustrated text: Pictures help to build mental models☆ , 1992 .

[21]  M. Tanenhaus,et al.  Circumscribing Referential Domains during Real-Time Language Comprehension , 2002 .

[22]  E. André.  The Generation of Multimedia Presentations , 2000 .

[23]  Charles Goodwin,et al.  Pointing as Situated Practice , 1998 .

[24]  Sarah Boyd,et al.  Detecting and Describing Patterns in Time-Varying Data Using Wavelets , 1997, IDA.

[25]  Christopher Habel,et al.  Propositional and Depictorial Representations of Spatial Knowledge: The Case of Path-Concepts , 1989, Natural Language and Logic.

[26]  W. Levelt.  Speaking: From Intention to Articulation , 1990 .

[27]  Ellen M. Voorhees,et al.  Overview of the TREC 2004 Novelty Track. , 2005 .

[28]  Z. Obrenovic,et al.  Modeling multimodal human-computer interaction , 2004, Computer.

[29]  M. Tomasello,et al.  Understanding and sharing intentions: The origins of cultural cognition , 2005, Behavioral and Brain Sciences.

[30]  P. Cheng,et al.  Graph-based reasoning: from task analysis to cognitive explanation , 2001 .

[31]  I WilliamWinn.  Learning from Maps and Diagrams , 1991 .

[32]  Heiner Stuckenschmidt,et al.  Ontology-Based Integration of Information - A Survey of Existing Approaches , 2001, OIS@IJCAI.

[33]  Carlos Gussenhoven,et al.  Transcription of Dutch intonation , 2005 .

[34]  Download Book,et al.  Information Visualization in Data Mining and Knowledge Discovery , 2001 .

[35]  Mitsuru Ishizuka,et al.  Life-like characters - tools, affective functions, and applications , 2004, Life-like characters.

[36]  Darren Gergle.  What's There to Talk About? A Multi-Modal Model of Referring Behavior in the Presence of Shared Visual Information , 2006, EACL.

[37]  D. McNeill.  Hand and Mind , 1995 .

[38]  A. Kendon.  Some functions of gaze-direction in social interaction. , 1967, Acta psychologica.

[39]  J. I N Y U,et al.  Choosing the content of textual summaries of large time-series data sets , 2005 .

[40]  Leo G. M. Noordman,et al.  Demonstrative nominal anaphors: a case of nonidentificational markedness , 1995 .

[41]  Douglas J. Hacker,et al.  Environmental, cognitive, and metacognitive influences on text revision: Assessing the evidence , 1996 .

[42]  Mark Steedman,et al.  Prosodic marking of contrasts in information structure , 2006 .

[43]  Anna S. Law,et al.  A Comparison of Graphical and Textual Presentations of Time Series Data to Support Medical Decision Making in the Neonatal Intensive Care Unit , 2005, Journal of Clinical Monitoring and Computing.

[44]  Justine Cassell,et al.  External manifestations of trustworthiness in the interface , 2000, CACM.

[45]  S. Duncan,et al.  On signalling that it's your turn to speak☆ , 1974 .

[46]  J. Atkinson,et al.  A change-of-state token and aspects of its sequential placement , 1985 .

[47]  G. Butterworth.  Pointing Is the Royal Road to Language for Babies , 2003 .

[48]  Frank Marchak,et al.  Universal Design , 2001 .

[49]  Ahm Cremers,et al.  Dutch and English demonstratives : a comparison , 1996 .

[50]  Gerald L. Lohse,et al.  A Cognitive Model for Understanding Graphical Perception , 1993, Hum. Comput. Interact..

[51]  喜多 壮太郎.  Pointing : where language, culture, and cognition meet , 2013 .

[52]  R. Jackendoff.  Foundations of Language: Brain, Meaning, Grammar, Evolution , 2002 .

[53]  S. Duncan,et al.  On the structure of speaker–auditor interaction during speaking turns , 1974, Language in Society.

[54]  Vincent J. van Heuven,et al.  The significance of demonstrative position in modern Dutch , 1988 .

[55]  Robbert-Jan Beun,et al.  Multimodal Reference to Objects: An Empirical Approach , 1998, Cooperative Multimodal Communication.

[56]  B. Rimé.  The elimination of visible behaviour from social interactions: Effects on verbal, nonverbal and interpersonal variables , 1982 .

[57]  Emiel Krahmer,et al.  Audiovisual cues to uncertainty , 2003 .

[58]  Joakim Nivre,et al.  On the Semantics and Pragmatics of Linguistic Feedback , 1992, J. Semant..

[59]  Jenny Waycott,et al.  Effects of advance organizers, mental models and abilities on task and recall performance using a mobile phone network , 2006 .

[60]  Barbara J. Grosz,et al.  The representation and use of focus in dialogue understanding. , 1977 .

[61]  Anita Cremers,et al.  Reference to objects : an empirically based study of task-oriented dialogues , 1996 .

[62]  Jessica Lin,et al.  Finding Motifs in Time Series , 2002, KDD 2002.

[63]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[64]  De Ruiter,et al.  Gesture and speech production , 1998 .

[65]  Mary Ellen Foster.  Corpus-Based Planning of Deictic Gestures in COMIC , 2004, INLG.

[66]  Lars Kulik,et al.  Lexical Specifications of Paths , 2000, Spatial Cognition.

[67]  C. Goodwin.  Conversational Organization: Interaction Between Speakers and Hearers , 1981 .

[68]  Christopher Habel,et al.  Incremental generation of interconnected preverbal messages , 2004 .

[69]  María Luisa Flecha-García.  Eyebrow raising in dialogue: discourse structure, utterance function, and pitch accents , 2006 .

[70]  E. Schegloff.  Structures of Social Action: On some gestures' relation to talk , 1985 .

[71]  Frank J. Bernieri,et al.  The Importance of Nonverbal Cues in Judging Rapport , 1999 .

[72]  Evangelos Vlachogiannis,et al.  Universal access to information services—the need for user information and its relationship to device profiles , 2004, Universal Access in the Information Society.

[73]  Deborah L. McGuinness,et al.  IWTrust: Improving User Trust in Answers from the Web , 2005, iTrust.

[74]  Harry Bunt,et al.  Multimodal referece. Studies in automatic generation of multimodal referring expressions , 2000 .

[75]  S. Kosslyn.  Understanding charts and graphs , 1989 .

[76]  Brouwer-Janse,et al.  Referring in a shared workspace , 1994 .

[77]  M. Alibali,et al.  Transitions in concept acquisition: using the hand to read the mind. , 1993, Psychological review.

[78]  Adam Kendon,et al.  Gesticulation, Quotable Gestures, and Signs , 1990 .

[79]  J. D. Ruiter.  The production of gesture and speech , 2000 .

[80]  Helen Blood,et al.  Body Language and the Social Order: Communication as Behavioral Control , 1974 .

[81]  F. GANIER.  Factors affecting the processing of procedural instructions: implications for document design , 2004, IEEE Transactions on Professional Communication.

[82]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[83]  Christopher Habel,et al.  Prinzipien der Referentialität: Untersuchungen zur propositionalen Repräsentation von Wissen , 1986, Informatik-Fachberichte.

[84]  Lars Kulik,et al.  Shape Nouns and Shape Concepts: A Geometry for 'Corner' , 1998, Spatial Cognition.

[85]  N. Enfield.  ‘Lip-pointing’: A discussion of form and function with reference to data from Laos , 2002 .

[86]  U. Hadar,et al.  Head movement during listening turns in conversation , 1985 .

[87]  Richard L. Lewis,et al.  An Activation-Based Model of Sentence Processing as Skilled Memory Retrieval , 2005, Cogn. Sci..

[88]  A. Kendon.  Do Gestures Communicate? A Review , 1994 .

[89]  Emiel Krahmer,et al.  The influence of target size and distance on the production of speech and gesture in multimodal referring expressions , 2004, INTERSPEECH.

[90]  Eduard H. Hovy,et al.  On the Knowledge Underlying Multimedia Presentations , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[91]  Ehud Reiter,et al.  Generating Feedback Reports for Adults Taking Basic Skills Tests , 2005, SGAI Conf..

[92]  Anton Nijholt,et al.  Questions, pictures, answers : Introducing pictures in question-answering systems , 2007 .

[93]  Dave Pratt,et al.  The construction fo meanings for trend in active graphing , 2000, Int. J. Comput. Math. Learn..

[94]  Markus Guhe,et al.  The Influence of Resource Parameters on Incremental Conceptualization , 2001 .

[95]  Yuval Shahar,et al.  A Framework for Knowledge-Based Temporal Abstraction , 1997, Artif. Intell..

[96]  Robert M. Krauss,et al.  Gesture and Speech in Spontaneous and Rehearsed Narratives , 1994 .

[97]  Dipankar Dasgupta,et al.  Novelty detection in time series data using ideas from immunology , 1996 .

[98]  Emiel Krahmer,et al.  Generating Referring Expressions in a Multimodal Context An empirically oriented approach , 2000, CLIN.

[99]  S. Brennan.  Centering Attention in Discourse. , 1995 .

[100]  Werner Deutsch,et al.  The Development of Verbal and Nonverbal Devices for Reference. , 1982 .

[101]  Wim Claassen,et al.  Generating Referring Expressions in a Multimodal Environment , 1992, NLG.

[102]  H. H. Clark,et al.  Changing Ideas about Reference , 2004 .

[103]  P. Fitts.  The information capacity of the human motor system in controlling the amplitude of movement. , 1954, Journal of experimental psychology.

[104]  Steven Pinker,et al.  A theory of graph comprehension. , 1990 .

[105]  Starkey D. Duncan.  Language, Paralanguage, and Body Motion in the Structure of Conversations , 1975 .

[106]  T. Trabasso,et al.  Offering a Hand to Pragmatic Understanding: The Role of Speech and Gesture in Comprehension and Memory , 1999 .

[107]  Richard Reviewer-Granger.  Unified Theories of Cognition , 1991, Journal of Cognitive Neuroscience.

[108]  Marc Alexa,et al.  Visualizing time-series on spirals , 2001, IEEE Symposium on Information Visualization, 2001. INFOVIS 2001..

[109]  Christopher Habel,et al.  Abstract Structures in Spatial Cognition , 1997, Foundations of Computer Science: Potential - Theory - Cognition.

[110]  Richard Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[111]  Siobhan Chapman.  Logic and Conversation , 2005 .

[112]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[113]  Michael Argyle,et al.  The central Europe experiment: Looking at persons and looking at objects , 1976 .

[114]  Allison Woodruff,et al.  Guidelines for using multiple views in information visualization , 2000, AVI '00.

[115]  E. Goffman,et al.  Forms of talk , 1982 .

[116]  Herbert A. Simon,et al.  CaMeRa: A Computational Model of Multiple Representations , 1997, Cogn. Sci..

[117]  J. Cassell,et al.  Modeling Gaze Behavior as a Function of Discourse Structure , 1998 .

[118]  Mats Rooth.  A theory of focus interpretation , 1992, Natural Language Semantics.

[119]  Ehud Reiter,et al.  SumTime-Mousam: Configurable marine weather forecast generator , 2003 .

[120]  I. I. van der Sluis,et al.  Generating Referring Expressions in a Multimodal Context , 2001 .

[121]  Christopher Habel.  Cognitive Linguistics: The Processing of Spatial Concepts , 1988, LILOG-Report.

[122]  Stephen Michael Kosslyn,et al.  Graph Design for the Eye and Mind , 2006 .

[123]  Dirk Heylen,et al.  Experimenting with the Gaze of a Conversational Agent , 2002 .

[124]  Alistair G. Sutcliffe,et al.  Task-related information analysis , 1997, Int. J. Hum. Comput. Stud..

[125]  Stacy Marsella,et al.  Natural Behavior of a Listening Agent , 2005, IVA.

[126]  Ehud Reiter,et al.  Lessons from a failure: Generating tailored smoking cessation letters , 2003, Artif. Intell..

[127]  Toni Rietveld,et al.  Statistical Techniques for the Study of Language and Language Behaviour , 1993 .

[128]  R. Krauss,et al.  Do conversational hand gestures communicate? , 1991, Journal of personality and social psychology.

[129]  Stefan Kopp,et al.  Synthese und Koordination von Sprache und Gestik für virtuelle multimodale Agenten , 2003, DISKI.

[130]  Norihiro Hagita,et al.  Messages embedded in gaze of interface agents --- impression management with agent's gaze , 2002, CHI.

[131]  Nigel G. Ward,et al.  Prosodic features which cue back-channel responses in English and Japanese , 2000 .

[132]  Yihsiu Chen,et al.  Language and Gesture: Lexical gestures and lexical access: a process model , 2000 .

[133]  Eamonn J. Keogh,et al.  Finding surprising patterns in a time series database in linear time and space , 2002, KDD.

[134]  Ben Shneiderman,et al.  LifeLines: using visualization to enhance navigation and analysis of patient records , 1998, AMIA.

[135]  H. H. Clark.  Arenas of language use , 1993 .

[136]  Richard L. Lewis,et al.  An Architecturally-based Theory of Human Sentence Comprehension , 2001 .

[137]  Rachelle S. Heller,et al.  Using a theoretical multimedia taxonomy framework , 2001, JERC.

[138]  A. Kendon,et al.  Pointing by Hand in “Neapolitan” , 2003 .

[139]  E. Schegloff,et al.  Two preferences in the organization of reference to persons in conversation and their interaction , 1979 .

[140]  Nicole Chovil.  Discourse‐oriented facial displays in conversation , 1991 .

[141]  Sanda M. Harabagiu,et al.  COGEX: A Logic Prover for Question Answering , 2003, NAACL.

[142]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[143]  Willem J. M. Levelt,et al.  Pointing and voicing in deictic expressions , 1985 .

[144]  Barbara Tversky,et al.  Animation: can it facilitate? , 2002, Int. J. Hum. Comput. Stud..

[145]  Emiel Krahmer,et al.  A New Model for Generating Multimodal Referring Expressions , 2003, ENLG@EACL.

[146]  Heather Shovelton,et al.  Why the spontaneous images created by the hands during talk can help make TV advertisements more effective. , 2005, British journal of psychology.

[147]  Robbert-Jan Beun,et al.  Object reference in a shared domain of conversation , 1998 .

[148]  A. Bangerter,et al.  Using Pointing and Describing to Achieve Joint Focus of Attention in Dialogue , 2004, Psychological science.

[149]  Charles S. Peirce,et al.  The Philosophy of Peirce : Selected Writings , 1941 .

[150]  Louis Vuurpijl,et al.  SLOT: A research platform for investigating multimodal communication , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[151]  Asli Ozyurek.  Do Speakers Design Their Cospeech Gestures for Their Addressees? The Effects of Addressee Location on Representational Gestures , 2002 .

[152]  Boris Reuderink,et al.  The Influence of Gaze and Head Tilt on the Impression of Listening Agents , 2006 .

[153]  Paul Piwek,et al.  Modality Choice for Generation of Referring Acts: Pointing versus Describing , 2007 .

[154]  James C. Lester,et al.  The persona effect: affective impact of animated pedagogical agents , 1997, CHI.

[155]  J. Hart,et al.  Memory and the feeling-of-knowing experience. , 1965, Journal of educational psychology.

[156]  B. Landau,et al.  “What” and “where” in spatial language and spatial cognition , 1993 .

[157]  Gary Ewing,et al.  NEONATE: Decision Support in the Neonatal Intensive Care Unit - A Preliminary Report , 2003, AIME.

[158]  Robert McGill,et al.  Graphical Perception: The Visual Decoding of Quantitative Information on Graphical Displays of Data , 1987 .

[159]  Lars Kulik,et al.  A Geometric Agent Following Route Instructions , 2003, Spatial Cognition.

[160]  H. H. Clark,et al.  Speaking while monitoring addressees for understanding , 2004 .

[161]  S. Duncan,et al.  Some Signals and Rules for Taking Speaking Turns in Conversations , 1972 .

[162]  Eamonn J. Keogh,et al.  HOT SAX: efficiently finding the most unusual time series subsequence , 2005, Fifth IEEE International Conference on Data Mining (ICDM'05).

[163]  V. Yngve.  On getting a word in edgewise , 1970 .

[164]  Johanna D. Moore,et al.  AutoBrief: an experimental system for the automatic generation of briefings in integrated text and information graphics , 2004, Int. J. Hum. Comput. Stud..

[165]  M. Alibali,et al.  Gesture-Speech Mismatch and Mechanisms of Learning: What the Hands Reveal about a Child′s State of Mind , 1993, Cognitive Psychology.

[166]  M. Hegarty,et al.  Graphs as aids to knowledge construction: Signaling techniques for guiding the process of graph comprehension. , 1999 .

[167]  Matthew Stone,et al.  Specifying and animating facial signals for discourse in embodied conversational agents , 2004, Comput. Animat. Virtual Worlds.

[168]  E. Schegloff.  Discourse as an interactional achievement : Some uses of "Uh huh" and other things that come between sentences , 1982 .

[169]  Edward Rolf Tufte,et al.  The visual display of quantitative information , 1985 .

[170]  Walter Daelemans,et al.  Learning to Predict Pitch Accents and Prosodic Boundaries in Dutch , 2003, ACL.

[171]  Ezequiel Morsella,et al.  The role of gestures in spatial working memory and speech. , 2004, The American journal of psychology.

[172]  E. Reiter,et al.  Acquiring Correct Knowledge for Natural Language Generation , 2011, J. Artif. Intell. Res..

[173]  Russell N. Carney,et al.  Pictorial Illustrations Still Improve Students' Learning from Text , 2002 .

[174]  M. Swerts,et al.  Audiovisual prosody and feeling of knowing , 2005 .

[175]  Carla Huls,et al.  Studies into Full Integration of Language and Action , 1995, Multimodal Human-Computer Communication.

[176]  H. H. Clark.  Pointing and placing. , 2003 .

[177]  Ariel Cohen,et al.  How Are Alternatives Computed? , 1999, J. Semant..

[178]  Jimmy J. Lin,et al.  What Makes a Good Answer? The Role of Context in Question Answering , 2003, INTERACT.

[179]  H. H. Clark,et al.  Audience Design in Meaning and Reference , 1982 .

[180]  Eamonn J. Keogh,et al.  An online algorithm for segmenting time series , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[181]  R McGill,et al.  Graphical Perception and Graphical Methods for Analyzing Scientific Data , 1985, Science.

[182]  R. Mayer,et al.  Aids to computer-based multimedia learning , 2002 .

[183]  S. Schwan,et al.  The cognitive benefits of interactive videos: learning to tie nautical knots , 2004 .

[184]  R. Krauss.  Why Do We Gesture When We Speak? , 1998 .

[185]  Mariët Theune,et al.  ANGELICA : choice of output modality in an embodied agent , 2001 .

[186]  William D. Marslen-Wilson,et al.  Producing Interpretable Discourse: The Establishment and Maintenance of Reference , 1982 .

[187]  A. Kendon.  Gesticulation and Speech: Two Aspects of the Process of Utterance , 1981 .

[188]  David P. Wilkins.  Why pointing with the index finger is not a universal (in sociocultural and semiotic terms). , 2003 .

[189]  Evelyn Z. McClave.  Linguistic functions of head movements in the context of speech , 2000 .

[190]  Robert E. Kraut,et al.  Visual Information as a Conversational Resource in Collaborative Physical Tasks , 2003, Hum. Comput. Interact..

[191]  Ipke Wachsmuth,et al.  Incremental Generation of Multimodal Deixis Referring to Objects , 2005, ENLG.

[192]  Pat Hanrahan,et al.  Identification and validation of cognitive design principles for automated generation of assembly instructions , 2004, AVI.

[193]  Joseph Bates,et al.  The role of emotion in believable agents , 1994, CACM.

[194]  G. Beattie.  Visible Thought: the new psychology of body language , 2007 .

[195]  R. Mayer.  Systematic Thinking Fostered by Illustrations in Scientific Text , 1989 .

[196]  Julio Abascal,et al.  Universal accessibility as a multimodal design issue , 2007, CACM.

[197]  Martin Kay,et al.  Syntactic Process , 1979, ACL.

[198]  Jim Hunter,et al.  Choosing words in computer-generated weather forecasts , 2005, Artif. Intell..

[199]  Ray Jackendoff,et al.  The Architecture of the Language Faculty , 1996 .

[200]  Wolfgang Wahlster,et al.  Plan-Based Integration of Natural Language and Graphics Generation , 1993, Artif. Intell..

[201]  A. Scheflen.  THE SIGNIFICANCE OF POSTURE IN COMMUNICATION SYSTEMS. , 1964, Psychiatry.

[202]  R. Krauss,et al.  The Communicative Value of Conversational Hand Gesture , 1995 .

[203]  M. Argyle,et al.  Gaze and Mutual Gaze , 1994, British Journal of Psychiatry.

[204]  Paul Piwek,et al.  Incremental Conceptualization for Language Production , 2020 .

[205]  A. Chaudhuri,et al.  The Many Faces of a Neutral Face: Head Tilt and Perception of Dominance and Emotion , 2003 .

[206]  S. Brennan,et al.  THE FEELING OF ANOTHER'S KNOWING : PROSODY AND FILLED PAUSES AS CUES TO LISTENERS ABOUT THE METACOGNITIVE STATES OF SPEAKERS , 1995 .

[207]  W. S. Condon,et al.  A segmentation of behavior , 1967 .

[208]  Kees van Deemter,et al.  The Clarity-Brevity Trade-off in Generating Referring Expressions , 2006, INLG.

[209]  Dirk Heylen,et al.  Head Gestures, Gaze and the Principles of Conversational Structure , 2006, Int. J. Humanoid Robotics.

[210]  Markus Guhe.  Marking theme and rheme in preverbal messages ∗ , 2006 .

[211]  Ehud Reiter,et al.  Book Reviews: Building Natural Language Generation Systems , 2000, CL.

[212]  L A Thompson,et al.  Evaluation and integration of speech and pointing gestures during referential understanding. , 1986, Journal of experimental child psychology.

[213]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[214]  M. Just,et al.  Constructing mental models of machines from text and diagrams. , 1993 .

[215]  Stephen M. Kosslyn,et al.  Elements of graph design , 1993 .

[216]  Sharon L. Oviatt,et al.  Toward a theory of organized multimodal integration patterns during human-computer interaction , 2003, ICMI '03.

[217]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987, Cogn. Sci..

[218]  Thies Pfeiffer,et al.  Measuring and Reconstructing Pointing in Visual Contexts , 2006 .

[219]  Dianne C. Berry,et al.  Learning a procedural task: effectiveness of multimedia presentations , 2000 .

[220]  Gaea Leinhardt,et al.  Functions, Graphs, and Graphing: Tasks, Learning, and Teaching , 1990 .

[221]  Emanuel A. Schegloff,et al.  Issues of Relevance for Discourse Analysis: Contingency in Action, Interaction and Co-Participant Context , 1996 .

[222]  D. Povinelli,et al.  Differences between chimpanzees (Pan troglodytes) and humans (Homo sapiens) in the resting state of the index finger: implications for pointing. , 1994, Journal of comparative psychology.

[223]  Klaus Krippendorff,et al.  Content Analysis: An Introduction to Its Methodology , 1980 .

[224]  Kristinn R. Thórisson,et al.  The Power of a Nod and a Glance: Envelope Vs. Emotional Feedback in Animated Conversational Agents , 1999, Appl. Artif. Intell..