Intrinsic Motion Stability Assessment for Video Stabilization

This paper presents a novel algorithm for assessing the motion stability of a video after stabilization. The assessment works in a non-reference manner that directly measures the intrinsic smoothness of the video motion path. Specifically, the motion path is cast as a curve embedded in the Lie group of homographies, and its smoothness is mathematically characterized by the intrinsic geodesic curvature. A bundle of paths are adopted to handle spatially variant motions through the frames. Then, we compute the weighted curvature for a holistic assessment on the motion stability. Other factors related to video stabilization, e.g., distortion and cropping, are also investigated as supplement. We collect 160 shaky video clips and their stabilized results for verification, and the experimental evidence shows the effectiveness of our algorithm in good correlation with human subjective judgements.

[1]  Ernesto Zacur,et al.  Left-Invariant Riemannian Geodesics on Spatial Transformation Groups , 2014, SIAM J. Imaging Sci..

[2]  Irfan A. Essa,et al.  Auto-directed video stabilization with robust L1 optimal camera paths , 2011, CVPR 2011.

[3]  Sylvain Paris,et al.  Example-Based Video Color Grading Supplemental material – Curvature estimation , 2013 .

[4]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[5]  Feng Liu,et al.  Spatially and Temporally Optimized Video Stabilization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[6]  Jian Sun,et al.  Bundled camera paths for video stabilization , 2013, ACM Trans. Graph..

[7]  Han Zhao,et al.  Simultaneous Camera Path Optimization and Distraction Removal for Improving Amateur Video , 2015, IEEE Transactions on Image Processing.

[8]  B. Hall.  Lie Groups, Lie Algebras, and Representations: An Elementary Introduction , 2004 .

[9]  Hua Huang,et al.  A Global Approach to Fast Video Stabilization , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Xavier Pennec,et al.  Efficient Parallel Transport of Deformations in Time Series of Images: From Schild’s to Pole Ladder , 2013, Journal of Mathematical Imaging and Vision.

[11]  Olli Silvén,et al.  Video Stabilization Performance Assessment , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[12]  John M. Lee.  Riemannian Manifolds: An Introduction to Curvature , 1997 .

[13]  Shi-Min Hu,et al.  Robust background identification for dynamic video editing , 2016, ACM Trans. Graph..

[14]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[15]  Hsien-chung Wang,et al.  Discrete nilpotent subgroups of Lie groups , 1969 .

[16]  T. Banchoff,et al.  Differential Geometry of Curves and Surfaces , 2010 .

[17]  Hui Huang,et al.  Image recoloring using geodesic distance based color harmonization , 2015, Computational Visual Media.

[18]  Hua Huang,et al.  Geodesic Video Stabilization in Transformation Space , 2017, IEEE Transactions on Image Processing.

[19]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[20]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[21]  T. Caelli.  Visual Perception: Theory and Practice , 1981 .

[22]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[23]  Harry Shum,et al.  Full-frame video stabilization , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Jin Wei,et al.  Timeline Editing of Objects in Video , 2013, IEEE Transactions on Visualization and Computer Graphics.

[25]  Sebastiano Battiato,et al.  SIFT Features Tracking for Video Stabilization , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[26]  Li Song,et al.  Shaking video synthesis for video stabilization performance assessment , 2013, 2013 Visual Communications and Image Processing (VCIP).

[27]  R. Howe.  Very Basic Lie Theory , 1983 .