Auto-directed video stabilization with robust L1 optimal camera paths

We present a novel algorithm for automatically applying constrainable, L1-optimal camera paths to generate stabilized videos by removing undesired motions. Our goal is to compute camera paths that are composed of constant, linear and parabolic segments mimicking the camera motions employed by professional cinematographers. To this end, our algorithm is based on a linear programming framework to minimize the first, second, and third derivatives of the resulting camera path. Our method allows for video stabilization beyond the conventional filtering of camera paths that only suppresses high frequency jitter. We incorporate additional constraints on the path of the camera directly in our algorithm, allowing for stabilized and retargeted videos. Our approach accomplishes this without the need of user interaction or costly 3D reconstruction of the scene, and works as a post-process for videos from any camera or from an online source.

[1]  Ariel Shamir,et al.  Improved seam carving for video retargeting , 2008, SIGGRAPH 2008.

[2]  Michael Gleicher,et al.  Re-cinematography: Improving the camerawork of casual video , 2008, TOMCCAP.

[3]  Hailin Jin,et al.  Light field video stabilization , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[4]  Stephen P. Boyd,et al.  1 Trend Filtering , 2009, SIAM Rev..

[5]  Mei Han,et al.  Discontinuous seam-carving for video retargeting , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Harry Shum,et al.  Full-frame video stabilization with motion inpainting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Michal Irani,et al.  Multi-Frame Correspondence Estimation Using Subspace Constraints , 2002, International Journal of Computer Vision.

[8]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[9]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[10]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Michael Gleicher,et al.  Video retargeting: automating pan and scan , 2006, MM '06.

[12]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[13]  Hans-Peter Seidel,et al.  Motion-aware temporal coherence for video resizing , 2009, ACM Trans. Graph..

[14]  Michael Bosse,et al.  Non-metric image-based rendering for video stabilization , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.