Full-Reference Stability Assessment of Digital Video Stabilization Based on Riemannian Metric

Assessing the quality of the motion stability is important to evaluating the performance of video stabilization algorithms. This paper presents a novel quality assessment scheme for the video motion stability in a full-reference (FR) manner. Given ideally stable videos and their corresponding shaky videos, our method measures the geodesic distance between motion paths of the stable and the stabilized videos. Due to the use of the Riemannian metric defined on the manifold of spatial transformations, our method enables the intrinsic and faithful measurement on pairwise motion disparities. To facilitate the FR assessment, a data set of stable and shaky videos is constructed by directly capturing realistic stable/shaky videos with a customized device. Then, digital video stabilization algorithms can be run on shaky videos to obtain the stabilized sequence of frames, whereupon their performances are evaluated by using our stability assessment. The experiments demonstrate that our stability assessment gains good concordance with the subjective assessment.

[1]  Li Song,et al.  Shaking video synthesis for video stabilization performance assessment , 2013, 2013 Visual Communications and Image Processing (VCIP).

[2]  Irfan A. Essa,et al.  Auto-directed video stabilization with robust L1 optimal camera paths , 2011, CVPR 2011.

[3]  Zihan Zhou,et al.  Robust plane-based structure from motion , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Alan C. Bovik,et al.  Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos , 2010, IEEE Transactions on Image Processing.

[5]  Rama Chellappa,et al.  Performance Characterization of Image Stabilization Algorithms , 1996, Real Time Imaging.

[6]  Ernesto Zacur,et al.  Left-Invariant Riemannian Geodesics on Spatial Transformation Groups , 2014, SIAM J. Imaging Sci..

[7]  Gianni Vernazza,et al.  Image stabilization algorithms for video-surveillance applications , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[8]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[9]  Shi-Min Hu,et al.  Robust background identification for dynamic video editing , 2016, ACM Trans. Graph..

[10]  Eli Peli,et al.  Motion perception during involuntary eye vibration , 2003, Experimental Brain Research.

[11]  Raanan Fattal,et al.  Video stabilization using epipolar geometry , 2012, TOGS.

[12]  S. Helgason.  Differential Geometry, Lie Groups, and Symmetric Spaces , 1978 .

[13]  Olli Silvén,et al.  Video Stabilization Performance Assessment , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[14]  Zhou Wang,et al.  Modern Image Quality Assessment , 2006, Modern Image Quality Assessment.

[15]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[16]  T. Caelli.  Visual Perception: Theory and Practice , 1981 .

[17]  Han Zhao,et al.  Simultaneous Camera Path Optimization and Distraction Removal for Improving Amateur Video , 2015, IEEE Transactions on Image Processing.

[18]  Pascal Vasseur,et al.  Homography Based Egomotion Estimation with a Common Direction , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Feng Liu,et al.  Spatially and Temporally Optimized Video Stabilization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[20]  Alan C. Bovik,et al.  Spatio-temporal quality pooling accounting for transient severe impairments and egomotion , 2011, 2011 18th IEEE International Conference on Image Processing.

[21]  Ali Borji,et al.  Salient Object Detection: A Benchmark , 2015, IEEE Transactions on Image Processing.

[22]  Shi-Min Hu,et al.  Hyper-Lapse From Multiple Spatially-Overlapping Videos , 2018, IEEE Transactions on Image Processing.

[23]  Damon M. Chandler,et al.  A spatiotemporal most-apparent-distortion model for video quality assessment , 2011, 2011 18th IEEE International Conference on Image Processing.

[24]  Sebastiano Battiato,et al.  SIFT Features Tracking for Video Stabilization , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[25]  Xuelong Li,et al.  Spatiotemporal Statistics for Video Quality Assessment , 2016, IEEE Transactions on Image Processing.

[26]  Thomas A. Funkhouser,et al.  A benchmark for 3D mesh segmentation , 2009, ACM Trans. Graph..

[27]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[28]  Hua Huang,et al.  A Global Approach to Fast Video Stabilization , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Christian Keimel.  Design of video quality metrics with multi-way data analysis , 2013 .

[30]  Chang-Su Kim,et al.  Video Stabilization Based on Feature Trajectory Augmentation and Selection and Robust Mesh Grid Warping , 2015, IEEE Transactions on Image Processing.

[31]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[32]  Rajiv Soundararajan,et al.  Video Quality Assessment by Reduced Reference Spatio-Temporal Entropic Differencing , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Jian Sun,et al.  Bundled camera paths for video stabilization , 2013, ACM Trans. Graph..

[34]  Moncef Gabbouj,et al.  Joint Video Stitching and Stabilization From Moving Cameras , 2016, IEEE Transactions on Image Processing.

[35]  Patrick Pérez,et al.  Time-sequential extraction of motion layers , 2008, 2008 15th IEEE International Conference on Image Processing.