Re-cinematography: improving the camera dynamics of casual video

This paper presents an approach to post-processing casually captured videos to improve apparent camera movement. Re-cinematography transforms each frame of a video such that the video better follows cinematic conventions. The approach breaks videos into shorter segments. For segments of the source video where the camera is relatively static, re-cinematography uses image stabilization to make the result look locked-down. For segments with camera motions, camera paths are keyframed automatically and interpolated with matrix logarithms to give velocity-profiled movements that appear intentional and directed. The approach automatically balances the tradeoff between motion smoothness and distortion to the original imagery. Results from our prototype show improvements to poor quality home videos.

[1]  Leonard McMillan,et al.  Proscenium: a framework for spatio-temporal video editing , 2003, ACM Multimedia.

[2]  Michal Irani,et al.  Video indexing based on mosaic representations , 1998, Proc. IEEE.

[3]  Patrick Olivier,et al.  Virtual Camera Planning: A Survey , 2005, Smart Graphics.

[4]  David Salesin,et al.  Multiperspective panoramas for cel animation , 1997, SIGGRAPH.

[5]  David Bordwell,et al.  Film Art: An Introduction , 1979 .

[6]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[7]  Xing Xie,et al.  A visual attention model for adapting images on small displays , 2003, Multimedia Systems.

[8]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[9]  Janusz Konrad,et al.  Probabilistic video stabilization using Kalman filtering and mosaicing , 2003, IS&T/SPIE Electronic Imaging.

[10]  Blain Brown.  Cinematography : theory and practice : imagemaking for cinematographers, directors & videographers , 2002 .

[11]  Michael Gleicher,et al.  Video retargeting: automating pan and scan , 2006, MM '06.

[12]  Alessandro Bevilacqua,et al.  High-Quality Real Time Motion Detection Using PTZ Cameras , 2006, 2006 IEEE International Conference on Video and Signal Based Surveillance.

[13]  L. Kavan,et al.  Dual Quaternions for Rigid Transformation Blending , 2006 .

[14]  R. Rosenholtz.  A simple saliency model predicts a number of motion popout phenomena , 1999, Vision Research.

[15]  Harry Shum,et al.  Full-frame video stabilization with motion inpainting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Michael Gleicher,et al.  Projective registration with difference decomposition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  F. Thomas,et al.  The illusion of life : Disney animation , 1981 .

[18]  Marc Alexa.  Linear Combinations of Transformations , 2002 .

[19]  P. Anandan,et al.  Mosaic based representations of video sequences and their applications , 1995, Proceedings of IEEE International Conference on Computer Vision.

[20]  Joan E. Hart,et al.  Film Directing Shot by Shot: Visualizing from Concept to Screen , 1991 .

[21]  Michael Bosse,et al.  Non-metric image-based rendering for video stabilization , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  David Nistér,et al.  Preemptive RANSAC for live structure and motion estimation , 2005, Machine Vision and Applications.

[23]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Shingo Uchihashi,et al.  A semi-automatic approach to home video editing , 2000, UIST '00.

[25]  John A. Robinson,et al.  Techniques for automated reverse storyboarding , 2005 .

[26]  E. Callenbach.  Grammar of the Film Language . Daniel Arjon. , 1993 .

[27]  Venu Madhav Govindu.  Lie-algebraic averaging for globally consistent motion estimation , 2004, CVPR 2004.

[28]  Marc Alexa,et al.  Linear combination of transformations , 2002, ACM Trans. Graph..

[29]  Michael Gleicher,et al.  Virtual videography , 2006, MM '06.

[30]  Benjamin B. Bederson,et al.  Automatic thumbnail cropping and its effectiveness , 2003, UIST '03.

[31]  Richard Szeliski,et al.  Video mosaics for virtual environments , 1996, IEEE Computer Graphics and Applications.

[32]  Michal Irani,et al.  Recovery of ego-motion using image stabilization , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Lie Lu,et al.  Optimization-based automated home video editing system , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Gilman E. S. Toombes,et al.  Preemptive RANSAC for Live Structure and Motion Estimation , 2003 .