A novel accuracy assessment model for video stabilization approaches based on background motion

In this paper, we propose a new accuracy measurement model for the video stabilization method based on background motion that can accurately measure the performance of the video stabilization algorithm. Undesired residual motion present in the video can quantitatively be measured by the pixel by pixel background motion displacement between two consecutive background frames. First of all, foregrounds are removed from a stabilized video, and then we find the two-dimensional flow vectors for each pixel separately between two consecutive background frames. After that, we calculate a Euclidean distance between these two flow vectors for each pixel one by one, which is regarded as a displacement of each pixel. Then a total Euclidean distance of each frame is averaged to get a mean displacement for each pixel, which is called mean displacement error, and finally we calculate the average mean displacement error. Our experimental results show the effectiveness of our proposed method.

[1]  Guillaume-Alexandre Bilodeau,et al.  SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity , 2015, IEEE Transactions on Image Processing.

[2]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[3]  Gunnar Farnebäck,et al.  Two-Frame Motion Estimation Based on Polynomial Expansion , 2003, SCIA.

[4]  Jian Sun,et al.  Bundled camera paths for video stabilization , 2013, ACM Trans. Graph..

[5]  A. Bovik,et al.  OBJECTIVE VIDEO QUALITY ASSESSMENT , 2003 .

[6]  Harry Shum,et al.  Full-frame video stabilization with motion inpainting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Irfan A. Essa,et al.  Auto-directed video stabilization with robust L1 optimal camera paths , 2011, CVPR 2011.

[8]  M. Rezaei,et al.  Camera Motion Modeling for Video Stabilization Performance Assessment , 2011, 2011 7th Iranian Conference on Machine Vision and Image Processing.

[9]  Richard Szeliski,et al.  Removing rolling shutter wobble , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Stefan Winkler,et al.  Issues in vision modeling for perceptual video quality assessment , 1999, Signal Process..

[11]  David Jacobs,et al.  CTSR 2011-03 Digital Video Stabilization and Rolling Shutter Correction using Gyroscopes , 2011 .

[12]  Chao Zhang,et al.  Qualitative Assessment of Video Stabilization and Mosaicking Systems , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[13]  Xiaoming Liu,et al.  Temporally Robust Global Motion Compensation by Keypoint-Based Congealing , 2016, ECCV.

[14]  Jiajun Bu,et al.  Video stabilization with a depth camera , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Olli Silvén,et al.  Video Stabilization Performance Assessment , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[16]  Xin Li,et al.  Digital Video Processing and Communications , 2011 .

[17]  Yue Wang,et al.  A Multi-scale Evaluation Method for Motion Filtering in Digital Image Stabilization , 2015, 2015 IEEE 27th International Conference on Tools with Artificial Intelligence (ICTAI).

[18]  Rama Chellappa,et al.  Evaluation of image stabilization algorithms , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[19]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[20]  Rogério Melo Kinape,et al.  A study of the most important image quality measures , 2003, Proceedings of the 25th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE Cat. No.03CH37439).

[21]  Jian Sun,et al.  SteadyFlow: Spatially Smooth Optical Flow for Video Stabilization , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Paul S. Fisher,et al.  Image quality measures and their performance , 1995, IEEE Trans. Commun..

[23]  Per-Erik Forssén,et al.  Rectifying rolling shutter video from hand-held devices , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Irfan A. Essa,et al.  Calibration-free rolling shutter removal , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[25]  Li Song,et al.  Shaking video synthesis for video stabilization performance assessment , 2013, 2013 Visual Communications and Image Processing (VCIP).

[26]  Hua Huang,et al.  Geodesic Video Stabilization in Transformation Space , 2017, IEEE Transactions on Image Processing.

[27]  Hiroshi Mitani,et al.  VHS camcorder with electronic image stabilizer , 1989 .

[28]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.