The director's lens: an intelligent assistant for virtual cinematography

We present the Director's Lens, an intelligent interactive assistant for crafting virtual cinematography using a motion-tracked hand-held device that can be aimed like a real camera. The system employs an intelligent cinematography engine that can compute, at the request of the filmmaker, a set of suitable camera placements for starting a shot. These suggestions represent semantically and cinematically distinct choices for visualizing the current narrative. In computing suggestions, the system considers established cinema conventions of continuity and composition along with the filmmaker's previous selected suggestions, and also his or her manually crafted camera compositions, by a machine learning component that adapts shot editing preferences from user-created camera edits. The result is a novel workflow based on interactive collaboration of human creativity with automated intelligence that enables efficient exploration of a wide range of cinematographic possibilities, and rapid production of computer-generated animated movies.

[1]  Nicolas Courty,et al.  A Cinematography System for Virtual Storytelling , 2003, International Conference on Virtual Storytelling.

[2]  D. Arijon,et al.  Grammar of Film Language , 1976 .

[3]  Mark O. Riedl,et al.  A Lightweight Intelligent Virtual Cinematography System for Machinima Production , 2007, AIIDE.

[4]  Steven M. Drucker,et al.  CamDroid: a system for implementing intelligent camera control , 1995, I3D '95.

[5]  David Salesin,et al.  The virtual cinematographer: a paradigm for automatic real-time camera control and directing , 1996, SIGGRAPH.

[6]  R. Michael Young,et al.  Real-time cinematic camera control for interactive narratives , 2005, ACE '05.

[7]  R. Michael Young,et al.  Cinematic discourse generation , 2009 .

[8]  Blain Brown.  Cinematography : theory and practice : imagemaking for cinematographers, directors & videographers , 2002 .

[9]  Bryan Peterson,et al.  Learning to See Creatively , 1988 .

[10]  B. Blain.  Cinematography: Theory and Practice , 2016 .

[11]  Svetha Venkatesh,et al.  Director in your pocket: holistic help for the hapless home videographer , 2004, MULTIMEDIA '04.

[12]  Robert Michael Young,et al.  Longboard: A Sketch Based Intelligent Storyboarding Tool for Creating Machinima , 2008, FLAIRS Conference.

[13]  Colin Ware,et al.  Exploration and virtual camera control in virtual three dimensional environments , 1990, I3D '90.

[14]  Thomas Strothotte,et al.  A Camera Engine for Computer Games: Managing the Trade‐Off Between Constraint Satisfaction and Frame Coherence , 2001, Comput. Graph. Forum.

[15]  Carlo H. Séquin,et al.  Visibility preprocessing for interactive walkthroughs , 1991, SIGGRAPH.

[16]  Patrick Olivier,et al.  A real-time cinematography system for interactive 3D environments , 2010, SCA '10.

[17]  Marc Christie,et al.  Modeling Camera Control with Constrained Hypertubes , 2002, CP.

[18]  William H. Bares,et al.  Virtual 3D camera composition from frame constraints , 2000, ACM Multimedia.

[19]  Steven K. Feiner,et al.  Apex: An Experiment in the Automated Creation of Pictorial Explanations , 1985, IEEE Computer Graphics and Applications.

[20]  Henri Casanova,et al.  Virtual cinematography of group scenes using hierarchical lines of actions , 2008, Sandbox '08.

[21]  Michael Gleicher,et al.  Re-cinematography: improving the camera dynamics of casual video , 2007, ACM Multimedia.

[22]  Esteban Walter Gonzalez Clua,et al.  Neuronal editor agent for scene cutting in game cinematography , 2010, CIE.