Optimizing for aesthetically pleasing quadrotor camera motion

In this paper we first contribute a large scale online study (N ≈ 400) to better understand aesthetic perception of aerial video. The results indicate that it is paramount to optimize smoothness of trajectories across all keyframes. However, for experts timing control remains an essential tool. Satisfying this dual goal is technically challenging because it requires giving up desirable properties in the optimization formulation. Second, informed by this study we propose a method that optimizes positional and temporal reference fit jointly. This allows to generate globally smooth trajectories, while retaining user control over reference timings. The formulation is posed as a variable, infinite horizon, contour-following algorithm. Finally, a comparative lab study indicates that our optimization scheme outperforms the state-of-the-art in terms of perceived usability and preference of resulting videos. For novices our method produces smoother and better looking results and also experts benefit from generated timings.

[1]  Rolf Findeisen,et al.  Model predictive path-following for constrained nonlinear systems , 2009, Proceedings of the 48h IEEE Conference on Decision and Control (CDC) held jointly with 2009 28th Chinese Control Conference.

[2]  Vijay Kumar,et al.  Trajectory generation and control for precise aggressive maneuvers with quadrotors , 2012, Int. J. Robotics Res..

[3]  Vijay Kumar,et al.  Minimum snap trajectory generation and control for quadrotors , 2011, 2011 IEEE International Conference on Robotics and Automation.

[4]  Marc Christie,et al.  Efficient composition for virtual camera control , 2012, SCA '12.

[5]  Patrick Olivier,et al.  Camera Control in Computer Graphics , 2006, Eurographics.

[6]  N. Hogan Adaptive control of mechanical impedance by coactivation of antagonist muscles , 1984 .

[7]  Steven M. Drucker,et al.  Intelligent Camera Control in a Virtual Environment , 1994 .

[8]  Etienne Burdet,et al.  On the analysis of movement smoothness , 2015, Journal of NeuroEngineering and Rehabilitation.

[9]  R. E. Carlson,et al.  Monotone Piecewise Cubic Interpolation , 1980 .

[10]  Andrew P. Witkin,et al.  Spacetime constraints , 1988, SIGGRAPH.

[11]  Anil V. Rao,et al.  Practical Methods for Optimal Control Using Nonlinear Programming , 1987 .

[12]  Pat Hanrahan,et al.  Generating dynamically feasible trajectories for quadrotor cameras , 2016, ACM Trans. Graph..

[13]  D. Mayne,et al.  Robust receding horizon control of constrained nonlinear systems , 1993, IEEE Trans. Autom. Control..

[14]  HilligesOtmar,et al.  Real-time planning for automated multi-view drone cinematography , 2017 .

[15]  Otmar Hilliges,et al.  Airways: Optimization-Based Planning of Quadrotor Trajectories according to High-Level User Goals , 2016, CHI.

[16]  Siddhartha S. Srinivasa,et al.  Physics-based motion retiming , 2006, SCA '06.

[17]  Tsai-Yen Li,et al.  Real-Time Camera Planning for Navigation in Virtual Environments , 2008, Smart Graphics.

[18]  Marc Christie,et al.  Intuitive and efficient camera control with the toric space , 2015, ACM Trans. Graph..

[19]  Pat Hanrahan,et al.  An interactive tool for designing quadrotor camera shots , 2015, ACM Trans. Graph..

[20]  Chris Manzie,et al.  Multi-axis model predictive contouring control , 2013, Int. J. Control.

[21]  Pat Hanrahan,et al.  Towards a Drone Cinematographer: Guiding Quadrotor Cameras using Visual Composition Principles , 2016, ArXiv.

[22]  John T. Betts,et al.  Practical Methods for Optimal Control and Estimation Using Nonlinear Programming , 2009 .

[23]  Chao-Hung Lin,et al.  Efficient camera path planning algorithm for human motion overview , 2011, Comput. Animat. Virtual Worlds.

[24]  Raffaello D'Andrea,et al.  A model predictive controller for quadrocopter state interception , 2013, 2013 European Control Conference (ECC).

[25]  Nicolas Pronost,et al.  Interactive Character Animation Using Simulated Physics: A State‐of‐the‐Art Review , 2012, Comput. Graph. Forum.

[26]  João Pedro Hespanha,et al.  Performance limitations in reference tracking and path following for nonlinear systems , 2008, Autom..

[27]  Quentin Galvane,et al.  Automated Cinematography with Unmanned Aerial Vehicles , 2016, WICED@Eurographics.

[28]  C. Karen Liu,et al.  Composition of complex optimal multi-character motions , 2006, SCA '06.

[29]  Marc Christie,et al.  The director's lens: an intelligent assistant for virtual cinematography , 2011, ACM Multimedia.

[30]  Alexander Domahidi,et al.  Real-Time Motion Planning for Aerial Videography With Real-Time With Dynamic Obstacle Avoidance and Viewpoint Optimization , 2017, IEEE Robotics and Automation Letters.

[31]  Charles Richter,et al.  Aggressive flight of fixed-wing and quadrotor aircraft in dense indoor environments , 2015, Int. J. Robotics Res..

[32]  Manfred Morari,et al.  Optimization‐based autonomous racing of 1:43 scale RC cars , 2015, ArXiv.

[33]  Otmar Hilliges,et al.  WYFIWYG: Investigating Effective User Support in Aerial Videography , 2018, ArXiv.

[34]  James Shippen,et al.  Biomechanical metrics of aesthetic perception in dance , 2015, Experimental Brain Research.

[35]  Alexander Domahidi,et al.  Real-time planning for automated multi-view drone cinematography , 2017, ACM Trans. Graph..

[36]  D. Arijon,et al.  Grammar of Film Language , 1976 .