The Generation of Multimedia Presentations

Multimedia systems—systems which employ several media such as text, graphics, animation and sound for the presentation of information—have become widely available during the last decade. The acceptance and usability of such systems is, however, substantially affected by their limited ability to present information in a flexible manner. As the need for flexibity grows, the manual creation of multimedia presentations becomes less and less feasible. While the automatic production of material for presentation is rarely addressed in the multimedia community, a considerable amount of research effort has been directed towards the automatic generation of natural language. The purpose of this chapter is to introduce techniques for the automatic production of multimedia presentations; these techniques draw upon lessons learned during the development of natural language

[1]  Jerry R. Hobbs.  Why Is Discourse Coherent , 1978 .

[2]  Graeme Hirst,et al.  Anaphora in Natural Language Understanding: A Survey , 1981, Lecture Notes in Computer Science.

[3]  James F. Allen.  Maintaining knowledge about temporal intervals , 1983, CACM.

[4]  Joseph E. Grimes,et al.  The Thread of Discourse , 1984 .

[5]  Jock D. Mackinlay,et al.  Automating the design of graphical presentations of relational information , 1986, TOGS.

[6]  Gary J. Anglin,et al.  On Empirically Validating Functions of Pictures in Prose , 1987 .

[7]  William C. Mann,et al.  RHETORICAL STRUCTURE THEORY: A THEORY OF TEXT ORGANIZATION , 1987 .

[8]  Stuart C. Shapiro,et al.  Intelligent Multi-Media Interface Technology , 1988, SGCH.

[9]  Steven F. Roth,et al.  Graphics and Natural Language as Components of Automatic Explanation , 1988, SGCH.

[10]  Norbert Reithinger,et al.  XTRA: A Natural-Language Access System to Expert Systems , 1989, Int. J. Man Mach. Stud..

[11]  Johanna D. Moore,et al.  Planning Text for Advisory Dialogues , 1989, ACL.

[12]  Johanna D. Moore,et al.  Pointing: A Way Toward Explanation Dialogue , 1990, AAAI.

[13]  Som Bandyopadhyay.  Towards an understanding of coherence in multimodal discourse , 1990 .

[14]  Guy Lapalme,et al.  Text generation , 1990 .

[15]  Cécile L. Paris,et al.  Generation and Explanation: Building an Explanation Facility for the Explainable Expert Systems Framework , 1991 .

[16]  Stuart C. Shapiro,et al.  Intelligent multi-media interface technology , 1991 .

[17]  Pierre Falzon,et al.  AN ARCHITECTURE FOR MULTIMODAL DIALOGUE , 1991 .

[18]  Wolfgang Wahlster,et al.  WIP: The Automatic Synthesis of Multimodal Presentations , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[19]  Henry A. Kautz,et al.  Integrating Metric and Qualitative Temporal Reasoning , 1991, AAAI.

[20]  Bradley A. Goodman.  Multimedia Explanations for Intelligent Training Systems , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[21]  Mark T. Maybury,et al.  Planning Multimedia Explanations Using Communicative Acts , 1991, AAAI Workshop on Intelligent Multimedia Interfaces.

[22]  Wolfgang Wahlster,et al.  Designing Illustrated Texts: How Language Production Is Influenced by Graphics Generation , 1991, EACL.

[23]  Steven K. Feiner,et al.  Automating the generation of coordinated multimedia explanations , 1991, Computer.

[24]  Oliviero Stock,et al.  Natural Language and Exploration of an Information Space: The ALFresco Interactive System , 1991, IJCAI.

[25]  Stephen M. Casner,et al.  Task-analytic approach to the automated design of graphic presentations , 1991, TOGS.

[26]  Eduard H. Hovy,et al.  Automatic Generation of Formatted Text , 1991, AAAI.

[27]  Peter Wazinski.  Generating Spatial Descriptions for Cross-modal References , 1992, ANLP.

[28]  Wim Claassen,et al.  Generating Referring Expressions in a Multimodal Environment , 1992, NLG.

[29]  Ulrich Thiel,et al.  Integrating Natural Language Components into Graphical Discourse , 1992, ANLP.

[30]  Beth Adelson,et al.  Evocative agents and multi-media interface design , 1992, CHI.

[31]  Norbert Reithinger,et al.  The Performance of an Incremental Generation Component for Multi-Modal Dialog Contributions , 1992, NLG.

[32]  John Levine,et al.  Automatic Generation of On-Line Documentation in the IDAS Project , 1992, ANLP.

[33]  Winfried Graf,et al.  Constraint-Based Graphical Layout of Multimodal Presentations , 1998, Advanced Visual Interfaces.

[34]  Susanne van Mulken,et al.  Structure and Rules in Automated Multimedia Presentation Planning , 1993, IJCAI.

[35]  Akikazu Takeuchi,et al.  Social Interaction: Multimodal Conversation with Social Agents , 1994, AAAI.

[36]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[37]  Carla Huls,et al.  EDWARD: full integration of language and action in a multimodal user interface , 1994, Int. J. Hum. Comput. Stud..

[38]  Dick C. A. Bulterman,et al.  The Amsterdam hypermedia model: adding time and context to the Dexter model , 1994, CACM.

[39]  Wolfgang Maass.  From Vision to Multimodal Communication: Incremental Route Descriptions , 1994 .

[40]  Thomas Rist,et al.  Referring To World Objects With Text And Pictures , 1994, COLING.

[41]  Elisabeth André,et al.  Ein planbasierter Ansatz zur Generierung multimedialer Präsentationen , 1995, DISKI.

[42]  Henry Lieberman,et al.  Letizia: An Agent That Assists Web Browsing , 1995, IJCAI.

[43]  Steven F. Roth,et al.  SageBook: searching data-graphics by content , 1995, CHI '95.

[44]  James C. Lester,et al.  Dynamically Sequencing an Animated Pedagogical Agent , 1997, AAAI/IAAI, Vol. 1.

[45]  Thomas Rist,et al.  Coping with Temporal Constraints in Multimedia Presentation Planning , 1996, AAAI/IAAI, Vol. 1.

[46]  Ingrid Zukerman,et al.  Constraint Propagation in a Cooperative Approach for Multimodal Presentation Planning , 1996, ECAI.

[47]  Scott Prevost.  An Information Structural Approach to Spoken Language Generation , 1996, ACL.

[48]  Chris Mellish,et al.  Sources of Flexibility in Dynamic Hypertext Generation , 1996, INLG.

[49]  Guy Lapalme,et al.  PostGraphe: A System for the Generation of Statistical Graphics and Text , 1996, INLG.

[50]  Winfried Graf,et al.  A bibliographical survey of constraint-based approaches to CAD, graphics, layout, visualization, and related topics , 1996, Knowl. Based Syst..

[51]  Thomas Rist,et al.  A standard reference model for intelligent multimedia presentation systems , 1997, Comput. Stand. Interfaces.

[52]  Steven K. Feiner,et al.  Negotiation for automated generation of temporal multimedia presentations , 1997, MULTIMEDIA '96.

[53]  T. Joachims.  WebWatcher : A Tour Guide for the World Wide Web , 1997 .

[54]  Thomas Rist,et al.  Generating Multimedia Presentations for RoboCup Soccer Games , 1997, RoboCup.

[55]  Johanna D. Moore,et al.  Integrating planning and task-based design for multimedia presentation , 1997, IUI '97.

[56]  Janet E. Cahn,et al.  Improvising Linguistic Style: Social and Affective Bases for Agents. , 1997 .

[57]  Elisabeth André,et al.  WebPersona: A life-like presentation agent for the World-Wide Web , 1997, IJCAI 1997.

[58]  Thomas Rist,et al.  Adding animated presentation agents to the interface , 1997, IUI '97.

[59]  Maarten van Dantzich,et al.  Lifelike computer characters: the persona project at Microsoft , 1997 .

[60]  Sangkyu Park,et al.  Multimodal user interfaces in the Open Agent Architecture , 1997, IUI '97.

[61]  Clark Elliott,et al.  Autonomous Agents as Synthetic Characters , 1998, AI Mag..

[62]  Robert Dale,et al.  Integrating natural language generation and hypertext to produce dynamic documents , 1998, Interact. Comput..

[63]  Thomas Rist,et al.  Combining Alternatives in the Multimedia presentation of Decision Support Information for Real-Time Control , 1998, Designing Effective and Usable Multimedia Systems.

[64]  Thomas Rist,et al.  Employing AI Methods to Control the Behavior of Animated Interface Agents , 1999, Appl. Artif. Intell..

[65]  W. Lewis Johnson,et al.  Animated Agents for Procedural Training in Virtual Reality: Perception, Cognition, and Motor Control , 1999, Appl. Artif. Intell..

[66]  James C. Lester,et al.  Deictic Believability: Coordinated Gesture, Locomotion, and Speech in Lifelike Pedagogical Agents , 1999, Appl. Artif. Intell..