An Objective Character Believability Evaluation Procedure for Multi-agent Story Generation Systems

The ability to generate narrative is of importance to computer systems that wish to use story effectively for entertainment, training, or education. One of the focuses of intelligent virtual agent research in general and story generation research in particular is how to make agents/characters more lifelike and compelling. However, one question that invariably comes up is: Is the generated story good? An easier question to tackle is whether a reader/viewer of a generated story perceives certain essential attributes such as causal coherence and character believability. Character believability is the perception that story world characters are acting according to their own beliefs, desires, and intentions. We present a novel procedure for objectively evaluating stories generated for multiple agents/characters with regard to character intentionality - an important aspect of character believability. The process transforms generated stories into a standardized model of story comprehension and then indirectly compares that representation to reader/viewer mental perceptions about the story. The procedure is illustrated by evaluating a narrative planning system, Fabulist.

[1]  Michael Lebowitz,et al.  Story-telling as planning and learning , 1985 .

[2]  James C. Lester,et al.  Narrative prose generation , 2001, Artif. Intell..

[3]  Daniel S. Weld,et al.  UCPOP: A Sound, Complete, Partial Order Planner for ADL , 1992, KR.

[4]  Robert Michael Young,et al.  An intent-driven planner for multi-agent story generation , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[5]  Nicolas Szilas,et al.  IDtension: a narrative engine for Interactive Drama , 2003 .

[6]  Philip R. Cohen,et al.  Intentions in Communication. , 1992 .

[7]  James R. Meehan,et al.  The Metanovel: Writing Stories by Computer , 1976, Outstanding Dissertations in the Computer Sciences.

[8]  R. Michael Young,et al.  Notes on the Use of Plan Structures in the Creation of Interactive Plot , 1999 .

[9]  A. Graesser,et al.  Question answering in the context of stories , 1991 .

[10]  Andrew Stern,et al.  Integrating Plot, Character and Natural Language Processing in the Interactive Drama Façade , 2003 .

[11]  Robert Michael Young,et al.  Comparing Cognitive and Computational Models of Narrative Structure , 2004, AAAI.

[12]  Joseph Bates,et al.  The role of emotion in believable agents , 1994, CACM.

[13]  Stephen John Turner,et al.  The Creative Process: A Computer Model of Storytelling and Creativity , 1994 .

[14]  Mark O. Riedl,et al.  Narrative generation: balancing plot and character , 2004 .

[15]  David S. Miall.  Experiencing narrative worlds: On the psychological activities of reading , 2000 .

[16]  Marc Cavazza,et al.  Planning characters' behaviour in interactive storytelling , 2002, Comput. Animat. Virtual Worlds.

[17]  Marc Cavazza,et al.  Exploring scalability of character-based storytelling , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[18]  Philip R. Cohen,et al.  Intentions in Communication , 1991, CL.

[19]  Johanna D. Moore,et al.  Towards a Principled Representation of Discourse Plans , 1994, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society.

[20]  Jaime G. Carbonell,et al.  Interactive drama, art and artificial intelligence , 2002 .

[21]  Philip R. Cohen,et al.  Comments on Vanderveken and on Cohen and Levesque , 2003 .