Camera Control in Computer Graphics

Recent progress in modelling, animation and rendering means that rich, high fidelity virtual worlds are found in many interactive graphics applications. However, the viewer's experience of a 3D world is dependent on the nature of the virtual cinematography, in particular, the camera position, orientation and motion in relation to the elements of the scene and the action. Camera control encompasses viewpoint computation, motion planning and editing. We present a range of computer graphics applications and draw on insights from cinematographic practice in identifying their different requirements with regard to camera control. The nature of the camera control problem varies depending on these requirements, which range from augmented manual control (semi‐automatic) in interactive applications, to fully automated approaches. We review the full range of solution techniques from constraint‐based to optimization‐based approaches, and conclude with an examination of occlusion management and expressiveness in the context of declarative approaches to camera control.

[1]  David Salesin,et al.  Declarative Camera Control for Automatic Cinematography , 1996, AAAI/IAAI, Vol. 1.

[2]  Steven K. Feiner,et al.  Computer graphics: principles and practice (2nd ed.) , 1990 .

[3]  Jonathan H. Pickering Intelligent camera planning for computer graphics , 2002 .

[4]  Patrizia Palamidese A Camera Motion Metaphor Based on Film Grammar , 1996, Comput. Animat. Virtual Worlds.

[5]  Frederick P. Brooks,et al.  Walkthrough—a dynamic graphics system for simulating virtual buildings , 1987, I3D '86.

[6]  William H. Bares,et al.  A Model for Constraint-Based Camera Planning , 2000 .

[7]  Steven M. Drucker,et al.  CamDroid: a system for implementing intelligent camera control , 1995, I3D '95.

[8]  Mark H. Overmars,et al.  Motion Planning for Camera Movements in Virtual Environments , 2003 .

[9]  Dani Lischinski,et al.  Automatic Camera Placement for Image‐Based Modeling , 2000, Comput. Graph. Forum.

[10]  Steven K. Feiner,et al.  Evaluation of visual balance for automated layout , 2004, IUI '04.

[11]  Mateu Sbert,et al.  Importance-Driven Focus of Attention , 2006, IEEE Transactions on Visualization and Computer Graphics.

[12]  Patrick Olivier,et al.  Visual Composition as Optimisation , 1999 .

[13]  J. Koenderink,et al.  The internal representation of solid shape with respect to vision , 1979, Biological Cybernetics.

[14]  Frank Jardillier,et al.  Screen‐Space Constraints for Camera Movements: the Virtual Cameraman , 1998, Comput. Graph. Forum.

[15]  Nicolas Courty,et al.  Animation référencée vision : de la tâche au comportement , 2002 .

[16]  Thomas Strothotte,et al.  A Camera Engine for Computer Games: Managing the Trade‐Off Between Constraint Satisfaction and Frame Coherence , 2001, Comput. Graph. Forum.

[17]  Dani Lischinski,et al.  Automatic camera placement for image-based modeling , 1999, Proceedings. Seventh Pacific Conference on Computer Graphics and Applications (Cat. No.PR00293).

[18]  James C. Lester,et al.  Realtime Constraint-Based Cinematography for Complex Interactive 3D Worlds , 1998, AAAI/IAAI.

[19]  Hiroshi Hosobe Hierarchical nonlinear constraint satisfaction , 2004, SAC '04.

[20]  John F. Hughes,et al.  User-guided composition effects for art-based rendering , 2001, I3D '01.

[21]  Mateu Sbert,et al.  Viewpoint Selection using Viewpoint Entropy , 2001, VMV.

[22]  Steven M. Drucker,et al.  Intelligent camera control for graphical environments , 1994 .

[23]  Andreas Butz,et al.  Animation with CATHI , 1997, AAAI/IAAI.

[24]  Joseph V. Mascelli The five C's of cinematography : motion picture filming techniques simplified , 1965 .

[25]  John M. Snyder,et al.  Interval analysis for computer graphics , 1992, SIGGRAPH.

[26]  Tsai-Yen Li,et al.  An intelligent user interface with motion planning for 3D navigation , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[27]  Michael Werman,et al.  On View Likelihood and Stability , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  David Salesin,et al.  The virtual cinematographer: a paradigm for automatic real-time camera control and directing , 1996, SIGGRAPH.

[29]  Joseph V. Maschelli,et al.  The Five C's of Cinematography , 1965 .

[30]  Michael Gleicher,et al.  Through-the-lens camera control , 1992, SIGGRAPH.

[31]  Carol O'Sullivan,et al.  Predicting and Evaluating Saliency for Simplified Polygonal Models , 2005, TAP.

[32]  Yishai A. Feldman,et al.  Knowledge-Based Cinematography and Its Applications , 2004, ECAI.

[33]  Charles R. Dyer,et al.  Visibility, occlusion, and the aspect graph , 1990, International Journal of Computer Vision.

[34]  Zhengrong Liang,et al.  Interactive Path Planning for Virtual Endoscopy , 1999 .

[35]  Andreas Butz,et al.  Anymation with CATHI , 1997, AAAI 1997.

[36]  Patrick Rives,et al.  A new approach to visual servoing in robotics , 1992, IEEE Trans. Robotics Autom..

[37]  Massimo Zancanaro,et al.  Automatic Video Composition , 2003, Smart Graphics.

[38]  Erik Reinhard,et al.  Artistic Composition for Image Creation , 2001, Rendering Techniques.

[39]  Steven M. Drucker,et al.  CINEMA: a system for procedural camera movements , 1992, I3D '92.

[40]  Colin Ware,et al.  Exploration and virtual camera control in virtual three dimensional environments , 1990, I3D '90.

[41]  Steven K. Feiner,et al.  Automated generation of intent-based 3D Illustrations , 1991, SIGGRAPH.

[42]  Marc Cavazza,et al.  Real-Time Camera Control for Interactive Storytelling , 2002, GAME-ON.

[43]  Jock D. Mackinlay,et al.  Rapid controlled movement through a virtual 3D workspace , 1990, SIGGRAPH.

[44]  E. Krupinski,et al.  The Role of Formal Art Training on Perception and Aesthetic Judgment of Art Compositions , 2017 .

[45]  Gregory D. Hager,et al.  Dynamic sensor planning in visual servoing , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[46]  James F. Blinn,et al.  Where am I? What am I looking at? (cinematography) , 1988, IEEE Computer Graphics and Applications.

[47]  Steffi Beckhaus,et al.  Dynamic potential fields for guided exploration in virtual environments , 2002 .

[48]  Michael J. Maher,et al.  Over-Constrained Systems , 1995, Lecture Notes in Computer Science.

[49]  David Salesin,et al.  Multiperspective panoramas for cel animation , 1997, SIGGRAPH.

[50]  Dimitri Plemenos,et al.  Methods and data structures for virtual world exploration , 2006, The Visual Computer.

[51]  Mateu Sbert,et al.  Automatic View Selection Using Viewpoint Entropy and its Application to Image‐Based Modelling , 2003, Comput. Graph. Forum.

[52]  Gérard Subsol Virtual Storytelling, Using Virtual Reality Technologies for Storytelling, Third International Conference, ICVS 2005, Strasbourg, France, November 30 - December 2, 2005, Proceedings , 2005, International Conference on Virtual Storytelling.

[53]  Marc Christie,et al.  A Constraint-Based Approach to Camera Path Planning , 2003, Smart Graphics.

[54]  Andrew S. Forsberg,et al.  UniCam—2D gestural camera controls for 3D environments , 1999, SI3D.

[55]  Sung Je Hong,et al.  Through-the-lens camera control with a simple jacobian matrix , 1995 .

[56]  Marc Christie,et al.  Modeling Camera Control with Constrained Hypertubes , 2002, CP.

[57]  William H. Bares,et al.  Virtual 3D camera composition from frame constraints , 2000, ACM Multimedia.

[58]  Nicolas Courty,et al.  Controlling a camera in a virtual environment , 2002, The Visual Computer.

[59]  Desney S. Tan,et al.  Exploring 3D navigation: combining speed-coupled flying with orbiting , 2001, CHI.

[60]  Satoru Kawai,et al.  A simple method for computing general position in displaying three-dimensional objects , 1988, Comput. Vis. Graph. Image Process..

[61]  Jean-Claude Latombe,et al.  Robot motion planning , 1970, The Kluwer international series in engineering and computer science.

[62]  Pere-Pau Vázquez,et al.  Way‐Finder: guided tours through complex walkthrough models , 2004, Comput. Graph. Forum.

[63]  Patrice Torguet,et al.  Virtual Storytelling Using Virtual Reality Technologies for Storytelling , 2001, Lecture Notes in Computer Science.

[64]  Christopher G. Healey,et al.  Assisted navigation for large information spaces , 2002, IEEE Visualization, 2002. VIS 2002..

[65]  Patrick Rives,et al.  A new approach to visual servoing in robotics , 1992, IEEE Trans. Robotics Autom..

[66]  M. Zancanaro,et al.  Using Cinematic Techniques in a Multimedia Museum Guide. , 2003 .

[67]  Dimitri Plemenos,et al.  Viewpoint quality and scene understanding , 2005, VAST.

[68]  Wolfgang Stuerzlinger,et al.  Imaging all Visible Surfaces , 1999, Graphics Interface.

[69]  Daniel Thalmann,et al.  Physically-based interactive camera motion control using 3D input devices , 1991 .

[70]  Tsai-Yen Li,et al.  An intelligent 3D user interface adapting to user control behaviors , 2004, IUI '04.

[71]  Luke S. Zettlemoyer,et al.  Task-sensitive cinematography interfaces for interactive 3D learning environments , 1998, IUI '98.

[72]  James C. Lester,et al.  Cinematographic User Models for Automated Realtime Camera Control in Dynamic 3D Environments , 1997 .

[73]  Ernest Callenbach : The Five C's of Cinematography: Motion Picture Filming Techniques Simplified . Joseph V. Mascelli. , 1966 .

[74]  William D. Smart,et al.  Say Cheese! Experiences with a Robot Photographer , 2004, AI Mag..

[75]  George W. Fitzmaurice,et al.  HoverCam: interactive 3D navigation for proximal object inspection , 2005, I3D '05.

[76]  Patrizia Palamidese A Camera Motion Metaphor Based on Film Grammar , 1996 .

[77]  Dinesh Manocha,et al.  Interactive navigation in complex environments using path planning , 2003, I3D '03.

[78]  Marc Christie,et al.  Interval constraint solving for camera control and motion planning , 2000, TOCL.

[79]  George W. Fitzmaurice,et al.  StyleCam: interactive stylized 3D navigation using integrated spatial & temporal controls , 2002, UIST '02.

[80]  Dirk Bartz,et al.  Virtual voyage: interactive navigation in the human colon , 1997, SIGGRAPH.

[81]  Patrick Olivier,et al.  CamPlan: A Camera Planning Agent , 2000 .

[82]  Donghyun Kim,et al.  A camera control interface based on the visualization of subspaces of the 6D motion space of the camera , 1998, Proceedings Pacific Graphics '98. Sixth Pacific Conference on Computer Graphics and Applications (Cat. No.98EX208).

[83]  Steven K. Feiner,et al.  Supporting interactivity in automated 3D illustrations , 1993, IUI '93.

[84]  Nicolas Courty,et al.  Computer animation: a new application for image-based visual servoing , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[85]  Joan E. Hart,et al.  Film Directing Shot by Shot: Visualizing from Concept to Screen , 1991 .

[86]  Wolfgang Straßer,et al.  A case study on automatic camera placement and motion for visualizing historical data , 2002, IEEE Visualization, 2002. VIS 2002..

[87]  Abdul Sattar,et al.  Applying Constraint Weighting to Autonomous Camera Control , 2005, AIIDE.

[88]  Tsai-Yen Li,et al.  An Interactive Camera Planning System for Automatic Cinematographer , 2005, 11th International Multimedia Modelling Conference.

[89]  Ken Shoemake,et al.  ARCBALL: a user interface for specifying three-dimensional orientation using a mouse , 1992 .

[90]  Roger J. Hubbold,et al.  Navigation guided by artificial force fields , 1998, CHI.

[91]  William H. Bares,et al.  Storyboard frame editing for cinematic composition , 2002, IUI '02.

[92]  Han-Wei Shen,et al.  View selection for volume rendering , 2005, VIS 05. IEEE Visualization, 2005..

[93]  Donald B. Johnson,et al.  Testbed Evaluation of Virtual Environment Interaction Techniques , 1999, Presence: Teleoperators & Virtual Environments.

[94]  James C. Lester,et al.  Intelligent multi-shot visualization interfaces for dynamic 3D worlds , 1998, IUI '99.

[95]  Oussama Khatib,et al.  Real-Time Obstacle Avoidance for Manipulators and Mobile Robots , 1985, Autonomous Robot Vehicles.

[96]  Norman I. Badler,et al.  Automatic viewing control for 3D direct manipulation , 1992, I3D '92.

[97]  Doug A. Bowman,et al.  Testbed Evaluation of Virtual Environment Interaction Techniques , 2001, Presence Teleoperators Virtual Environ..

[98]  Bruce Blumberg,et al.  Expressive autonomous cinematography for interactive virtual environments , 2000, AGENTS '00.

[99]  William H. Bares A Photographic Composition Assistant for Intelligent Virtual 3D Camera Systems , 2006, Smart Graphics.

[100]  Nicolas Courty,et al.  A Cinematography System for Virtual Storytelling , 2003, International Conference on Virtual Storytelling.

[101]  D. Arijon,et al.  Grammar of Film Language , 1976 .

[102]  Eric A. Wernert,et al.  Constrained 3D navigation with 2D controllers , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[103]  Steven M. Drucker,et al.  Intelligent Camera Control in a Virtual Environment , 1994 .