Cinematic Visual Discourse: Representation, Generation, and Evaluation

In this paper, we present the design, implementation, and evaluation of an end-to-end camera planning system called Darshak. Darshak automatically constructs cinematic narrative discourse of a given story in a 3-D virtual environment. It utilizes a hierarchical partial-order causal link (POCL) planning algorithm to generate narrative plans that contain story events and camera directives for filming them. Dramatic situation patterns, commonly used by writers of fictional narratives, are formalized as communicative plan operators that provide a basis for structuring the cinematic content of the story's visualization. The dramatic patterns are realized through abstract communicative operators that represent operations on a viewer's beliefs about the story and its telling. Camera shot compositions and transitions are defined in this plan-based framework as execution primitives. Darshak's performance is evaluated through a novel user study based on techniques used to evaluate existing cognitive models of narrative comprehension. Initial study reveals significant effect of the choice of visualization strategies on measured viewer comprehension. It further shows significant effect of Darshak's choice of visualization strategy on comprehension.

[1]  Daniel S. Weld,et al.  UCPOP: A Sound, Complete, Partial Order Planner for ADL , 1992, KR.

[2]  Elena Not,et al.  Automatic cinematography and multilingual NLG for generating video documentaries , 2005, Artif. Intell..

[3]  Johanna D. Moore,et al.  Decomposition and Causality in Partial-order Planning , 1994, AIPS.

[4]  Robert Michael Young,et al.  An intent-driven planner for multi-agent story generation , 2004, Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems, 2004. AAMAS 2004..

[5]  Paolo Traverso,et al.  Automated planning - theory and practice , 2004 .

[6]  Patrick Olivier,et al.  CamPlan: A Camera Planning Agent , 2000 .

[7]  S. Chatman.  Story and Discourse: Narrative Structure in Fiction and Film , 1980 .

[8]  Joseph V. Maschelli,et al.  The Five C's of Cinematography , 1965 .

[9]  Robert Michael Young,et al.  A Discourse Planning Approach to Cinematic Camera Control for Narratives in Virtual Environments , 2005, AAAI.

[10]  Luca Di Gaspero,et al.  Virtual Camera Composition with Particle Swarm Optimization , 2008, Smart Graphics.

[11]  David Salesin,et al.  Declarative Camera Control for Automatic Cinematography , 1996, AAAI/IAAI, Vol. 1.

[12]  Jonathan H. Pickering.  Intelligent camera planning for computer graphics , 2002 .

[13]  Michael Gleicher,et al.  Through-the-lens camera control , 1992, SIGGRAPH.

[14]  Johanna D. Moore,et al.  Towards a Principled Representation of Discourse Plans , 1994, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society.

[15]  James F. Blinn,et al.  Where am I? What am I looking at? (cinematography) , 1988, IEEE Computer Graphics and Applications.

[16]  Johanna D. Moore,et al.  Planning Text for Advisory Dialogues , 1989, ACL.

[17]  David Salesin,et al.  The virtual cinematographer: a paradigm for automatic real-time camera control and directing , 1996, SIGGRAPH.

[18]  Mark T. Maybury,et al.  Communicative Acts for Explanation Generation , 1992, Int. J. Man Mach. Stud..

[19]  R. Michael Young,et al.  Real-time cinematic camera control for interactive narratives , 2005, ACE '05.

[20]  James C. Lester,et al.  Narrative prose generation , 2001, Artif. Intell..

[21]  Abdul Sattar,et al.  Applying Constraint Weighting to Autonomous Camera Control , 2005, AIIDE.

[22]  WareColin,et al.  Exploration and virtual camera control in virtual three dimensional environments , 1990 .

[23]  Georges Polti.  The Thirty-Six Dramatic Situations , 1916 .

[24]  Candace L. Sidner,et al.  Attention, Intentions, and the Structure of Discourse , 1986, CL.

[25]  D. Arijon,et al.  Grammar of Film Language , 1976 .

[26]  Johanna D. Moore,et al.  Planning Text for Advisory Dialogues: Capturing Intentional and Rhetorical Information , 1993, CL.

[27]  James C. Lester,et al.  Cinematographic User Models for Automated Realtime Camera Control in Dynamic 3D Environments , 1997 .

[28]  Neil Yorke-Smith,et al.  Temporal Reasoning with Preferences and Uncertainty , 2003, IJCAI.

[29]  Patrick Olivier,et al.  Virtual Camera Planning: A Survey , 2005, Smart Graphics.

[30]  Frank Jardillier,et al.  Screen‐Space Constraints for Camera Movements: the Virtual Cameraman , 1998, Comput. Graph. Forum.

[31]  Earl D. Sacerdoti,et al.  The Nonlinear Nature of Plans , 1975, IJCAI.

[32]  Thomas Strothotte,et al.  A Camera Engine for Computer Games: Managing the Trade‐Off Between Constraint Satisfaction and Frame Coherence , 2001, Comput. Graph. Forum.

[33]  Marc Christie,et al.  A Semantic Space Partitioning Approach to Virtual Camera Composition , 2005, Comput. Graph. Forum.

[34]  A. Graesser,et al.  Question answering in the context of stories , 1991 .

[35]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[36]  Stefan Edelkamp,et al.  Automated Planning: Theory and Practice , 2007, Künstliche Intell..

[37]  MARK O. RIEDL,et al.  Character-focused Narrative Planning , 2003 .

[38]  Thomas Michael Vernieri.  A Web Services Approach to Generating and Using Plans in Configurable Execution Environments , 2006 .

[39]  William C. Mann,et al.  RHETORICAL STRUCTURE THEORY: A THEORY OF TEXT ORGANIZATION , 1987 .

[40]  Robert Michael Young,et al.  Comparing Cognitive and Computational Models of Narrative Structure , 2004, AAAI.

[41]  William H. Bares,et al.  Virtual 3D camera composition from frame constraints , 2000, ACM Multimedia.

[42]  Bruce Blumberg,et al.  Expressive autonomous cinematography for interactive virtual environments , 2000, AGENTS '00.

[43]  Patrick Olivier,et al.  Camera Control in Computer Graphics , 2006, Eurographics.

[44]  James R. Meehan,et al.  TALE-SPIN, An Interactive Program that Writes Stories , 1977, IJCAI.

[45]  Colin Ware,et al.  Exploration and virtual camera control in virtual three dimensional environments , 1990, I3D '90.

[46]  James C. Lester,et al.  Intelligent multi-shot visualization interfaces for dynamic 3D worlds , 1998, IUI '99.

[47]  Dina Sherzer,et al.  A dictionary of narratology , 1989 .