Digital Video Stabilization in Static and Dynamic Scenes

The digital video stabilization is oriented on the removal of unintentional motions from video sequences caused by camera vibrations under external conditions, motion of robots stabilized platforms in a rugged landscape, a sea, oceans, or jitters during a non-professional hand-held shooting. The approaches for digital video stabilization in static and dynamic scenes are similar. However, objectively the analysis of dynamic scenes is needed in advanced intelligent methods. Several sequential stages include the choice of the key frames, the local and global motion estimations, the jitters compensation algorithm, the inpainting of frames boundaries, and the blurred frames restoration, for which the novel methods and algorithms were developed. The proposed application of fuzzy logic operators improves the separation results between the unwanted motion and the real motion of rigid objects. The corrective algorithm compensates the unwanted motion in frames; thereby the scene is aligned. The quality of stabilization in test video sequences was estimated by Peak Signal to Noise Ratio (PSNR) and Interframe Transformation Fidelity (ITF) metrics. During experiments, the PSNR and ITF estimations were received for six video sequences received from the static camera and eight video sequences received from the moving camera. The ITF estimations increase up on 3–4 dB or 15–20 % relative to the original video sequences.

[1]  Peyman Milanfar,et al.  Spatio-temporal video interpolation and denoising using motion-assisted steering kernel (MASK) regression , 2008, 2008 15th IEEE International Conference on Image Processing.

[2]  Sung Wook Baik,et al.  Adaptive key frame extraction for video summarization using an aggregation mechanism , 2012, J. Vis. Commun. Image Represent..

[3]  Du-Ming Tsai,et al.  Defect detection in periodically patterned surfaces using independent component analysis , 2008, Pattern Recognit..

[4]  Dan Schonfeld,et al.  Robust Video Stabilization Based on Particle Filter Tracking of Projected Camera Motion , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Jian-Feng Cai,et al.  Blind motion deblurring using multiple images , 2009, J. Comput. Phys..

[6]  Weijian Cong,et al.  Hybrid-template adaptive motion estimation algorithm based on block matching , 2010, 2010 International Conference on Computer and Communication Technologies in Agriculture Engineering.

[7]  Wen Gao,et al.  Online selecting discriminative tracking features using particle filter , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[8]  Stefania Colonnese,et al.  Markov Random Fields using complex line process: An application to Bayesian image restoration , 2011, 3rd European Workshop on Visual Information Processing.

[9]  Sheng Liu,et al.  A survey of fuzzy control for stabilized platforms , 2011, ArXiv.

[10]  Ioannis Andreadis,et al.  Digital Image Stabilization by Independent Component Analysis , 2010, IEEE Transactions on Instrumentation and Measurement.

[11]  Juan Moreno García,et al.  Video sequence motion tracking by fuzzification techniques , 2010, Appl. Soft Comput..

[12]  Yong Ho Kim,et al.  Toward a conceptual framework of key-frame extraction and storyboard display for video summarization , 2010 .

[13]  Qixiang Ye,et al.  Combined feature evaluation for adaptive visual object tracking , 2011, Comput. Vis. Image Underst..

[14]  Truong Q. Nguyen,et al.  Markov Random Field Model-Based Edge-Directed Image Interpolation , 2007, IEEE Transactions on Image Processing.

[15]  Sang Uk Lee,et al.  Adaptive large window correlation for optical flow estimation with discrete optimization , 2013, Image Vis. Comput..

[16]  Stefania Colonnese,et al.  Bayesian image interpolation using Markov random fields driven by visually relevant image features , 2013, Signal Process. Image Commun..

[17]  Richard Szeliski,et al.  Seamless Image Stitching of Scenes with Large Motions and Exposure Differences , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Werner X. Schneider,et al.  Classification of multiscale spatiotemporal energy features for video segmentation and dynamic objects prioritisation , 2013, Pattern Recognit. Lett..

[19]  Sebastiano Battiato,et al.  A Robust Block-Based Image/Video Registration Approach for Mobile Imaging Devices , 2010, IEEE Transactions on Multimedia.

[20]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[21]  Michio Sugeno,et al.  Industrial Applications of Fuzzy Control , 1985 .

[22]  P. R. Anurenjan,et al.  Video stabilization using Speeded Up Robust Features , 2011, 2011 International Conference on Communications and Signal Processing.

[23]  Zongmin Wu,et al.  Applying multiquadric quasi-interpolation for boundary detection , 2011, Comput. Math. Appl..

[24]  Gaofeng Meng,et al.  Image deblurring with matrix regression and gradient evolution , 2012, Pattern Recognit..

[25]  Jitendra Malik,et al.  Object detection using a max-margin Hough transform , 2009, CVPR.

[26]  Harry Shum,et al.  Full-frame video stabilization with motion inpainting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Antonios Gasteratos,et al.  A Recursive Fuzzy System for Efficient Digital Image Stabilization , 2008, Adv. Fuzzy Syst..

[28]  Lakhmi C. Jain,et al.  Recent Advances in Knowledge-based Paradigms and Applications , 2014 .

[29]  Hyuk-Jae Lee,et al.  Probabilistic Global Motion Estimation Based on Laplacian Two-Bit Plane Matching for Fast Digital Image Stabilization , 2008, EURASIP J. Adv. Signal Process..

[30]  M. Faouzi Belbachir,et al.  Motion estimation and compensation from noisy image sequences: A new filtering scheme , 2007, Image Vis. Comput..

[31]  Amit K. Roy-Chowdhury,et al.  Vector field analysis for multi-object behavior modeling , 2013, Image Vis. Comput..

[32]  Limin Ma,et al.  Approximation to the k-th derivatives by multiquadric quasi-interpolation method , 2009, J. Comput. Appl. Math..

[33]  Zhiguo Cao,et al.  A real-time embedded architecture for SIFT , 2013, J. Syst. Archit..

[34]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[35]  Sarp Ertürk,et al.  Membership function adaptive fuzzy filter for image sequence stabilization , 2004, IEEE Transactions on Consumer Electronics.

[36]  Bryan W. Scotney,et al.  Finite Element Laplacian Feature Detector , 2011, MVA.

[37]  D. Aboutajdine,et al.  Modeling of Video Sequences by Gaussian Mixture: Application in Motion Estimation by Block Matching Method , 2010, EURASIP J. Adv. Signal Process..

[38]  Hyung-Il Choi,et al.  Real Time Face Tracking with Pyramidal Lucas-Kanade Feature Tracker , 2007, ICCSA.

[39]  Jean Ponce,et al.  Non-uniform Deblurring for Shaken Images , 2010, International Journal of Computer Vision.

[40]  Vincent Lepetit,et al.  Fast Keypoint Recognition Using Random Ferns , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Shyh-Kang Jeng,et al.  Augmented keyframe , 2010, J. Vis. Commun. Image Represent..

[42]  Horst Bischof,et al.  On-line Random Forests , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[43]  Margarita N. Favorskaya,et al.  Early Smoke Detection in Outdoor Space by Spatio-temporal Clustering using a Single Video Camera , 2012, KES.

[44]  Wen-Hsien Fang,et al.  Markov model fuzzy-reasoning based algorithm for fast block motion estimation , 2006, J. Vis. Commun. Image Represent..

[45]  M. H. Shakoor,et al.  Statistical digital image stabilization , 2011 .

[46]  Francesc Moreno-Noguer,et al.  Efficient rotation invariant object detection using boosted Random Ferns , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  W. David Pan,et al.  Fast and accurate global motion estimation algorithm using pixel subsampling , 2008, Inf. Sci..

[48]  Horst Bischof,et al.  PROST: Parallel robust online simple tracking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[49]  Sarp Erturk,et al.  Real-Time Digital Image Stabilization Using Kalman Filters , 2002 .

[50]  Rosa Rodriguez-Sánchez,et al.  Image inpainting with nonsubsampled contourlet transform , 2013, Pattern Recognit. Lett..

[51]  Margarita N. Favorskaya,et al.  Accurate spatio-temporal reconstruction of missing data in dynamic scenes , 2013, Pattern Recognit. Lett..

[52]  Janusz Konrad,et al.  Probabilistic video stabilization using Kalman filtering and mosaicing , 2003, IS&T/SPIE Electronic Imaging.

[53]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Sebastiano Battiato,et al.  A robust video stabilization system by adaptive motion vectors filtering , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[55]  Horst W. Haussecker,et al.  Video stabilization to a global 3D frame of reference by fusing orientation sensor and image alignment data , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[56]  Irfan Essa,et al.  Texture optimization for example-based synthesis , 2005, SIGGRAPH 2005.

[57]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[58]  Sung Wook Baik,et al.  Weighting low level frame difference features for key frame extraction using Fuzzy comprehensive evaluation and indirect feedback relevance mechanism , 2011 .

[59]  Lu Wang,et al.  Single-Image Blind Deblurring for Non-uniform Camera-Shake Blur , 2012, ACCV.

[60]  Shang-Hong Lai,et al.  A robust real-time video stabilization algorithm , 2006, J. Vis. Commun. Image Represent..

[61]  Jyoti Singhai,et al.  Review of Motion Estimation and Video Stabilization techniques For hand held mobile video , 2011 .

[62]  Yu-Chun Peng,et al.  Integration of Digital Stabilizer With Video Codec for Digital Video Cameras , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[63]  Margarita N. Favorskaya,et al.  Motion Estimations based on Invariant Moments for Frames Interpolation in Stereovision , 2013, KES.

[64]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[65]  Mehdi Rezaei,et al.  Digital video stabilization system by adaptive fuzzy filtering , 2011, 2011 19th European Signal Processing Conference.

[66]  Stefania Colonnese,et al.  Fast image interpolation using Circular Harmonic Functions , 2010, 2010 2nd European Workshop on Visual Information Processing (EUVIP).

[67]  Yiannis Kompatsiaris,et al.  Robust Temporal Activity Templates Using Higher Order Statistics , 2009, IEEE Transactions on Image Processing.

[68]  Kamesh Namuduri,et al.  Video stabilization using principal component analysis and scale invariant feature transform in particle filter framework , 2009, IEEE Transactions on Consumer Electronics.

[69]  Moshe Porat,et al.  On texture and image interpolation using Markov models , 2009, Signal Process. Image Commun..

[70]  Yeon Ju Lee,et al.  Nonlinear Image Upsampling Method Based on Radial Basis Function Interpolation , 2010, IEEE Transactions on Image Processing.

[71]  Bruce A. Draper,et al.  Using a Product Manifold distance for unsupervised action recognition , 2012, Image Vis. Comput..

[72]  Georgios Tziritas,et al.  Equivalent Key Frames Selection Based on Iso-Content Principles , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[73]  Fabrice Morestin,et al.  A method for vector displacement estimation with ultrasound imaging and its application for thyroid nodular disease , 2008, Medical Image Anal..

[74]  Bin Liao,et al.  Color optical flow estimation based on gradient fields with extended constraints , 2010, 2010 International Conference on Networking and Information Technology.

[75]  Horst Bischof,et al.  On-Line Random Naive Bayes for Tracking , 2010, 2010 20th International Conference on Pattern Recognition.

[76]  Borko Furht,et al.  Encyclopedia of Multimedia , 2006 .

[77]  Brendan J. Frey,et al.  Video Epitomes , 2005, CVPR.

[78]  Qixiang Ye,et al.  Online feature evaluation for object tracking using Kalman Filter , 2008, 2008 19th International Conference on Pattern Recognition.

[79]  Wenbin Chen,et al.  Video Stabilization Using Scale-Invariant Features , 2007, 2007 11th International Conference Information Visualization (IV '07).

[80]  Kristin J. Dana,et al.  Real-time scene stabilization and mosaic construction , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[81]  Xiangjun Zhang,et al.  Image Interpolation by Adaptive 2-D Autoregressive Modeling and Soft-Decision Estimation , 2008, IEEE Transactions on Image Processing.

[82]  Narciso García,et al.  Automatic Feature-Based Stabilization of Video with Intentional Motion through a Particle Filter , 2008, ACIVS.

[83]  Reza Hassanpour,et al.  Digital Video Stabilization Using Radon Transform , 2012, 2012 International Conference on Digital Image Computing Techniques and Applications (DICTA).

[84]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[85]  Joonki Paik,et al.  Real-time iterative framework of regularized image restoration and its application to video enhancement , 2003, Real Time Imaging.

[86]  Bo Hu,et al.  Robust and Accurate Object Tracking Under Various Types of Occlusions , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[87]  Sebastiano Battiato,et al.  Digital video stabilization through curve warping techniques , 2008, IEEE Transactions on Consumer Electronics.

[88]  Seungyong Lee,et al.  Handling outliers in non-blind image deconvolution , 2011, 2011 International Conference on Computer Vision.

[89]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[90]  Bohyung Han,et al.  Modeling and segmentation of floating foreground and background in videos , 2012, Pattern Recognit..

[91]  Jiri Matas,et al.  P-N learning: Bootstrapping binary classifiers by structural constraints , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[92]  Sebastiano Battiato,et al.  A Robust Image Alignment Algorithm for Video Stabilization Purposes , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[93]  Seok-Woo Jang,et al.  Adaptive robust estimation of affine parameters from block motion vectors , 2005, Image Vis. Comput..

[94]  Sangkeun Lee,et al.  Statistical region selection for robust image stabilization using feature-histogram , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[95]  Horst Bischof,et al.  Semi-supervised On-Line Boosting for Robust Tracking , 2008, ECCV.

[96]  J. Jokovic,et al.  Scene cut detection in video by using combination of spatial-temporal video characteristics , 2009, 2009 9th International Conference on Telecommunication in Modern Satellite, Cable, and Broadcasting Services.

[97]  Chi-Man Pun,et al.  Log-Polar Wavelet Energy Signatures for Rotation and Scale Invariant Texture Classification , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[98]  Horst Bischof,et al.  Hough-based tracking of non-rigid objects , 2011, 2011 International Conference on Computer Vision.

[99]  Sei-Wang Chen,et al.  Video stabilization for a hand-held camera based on 3D motion model , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[100]  Mehdi Rezaei,et al.  Digital video stabilizer by adaptive fuzzy filtering , 2012, EURASIP J. Image Video Process..

[101]  Hujun Bao,et al.  Video stabilization based on a 3D perspective camera model , 2009, The Visual Computer.

[102]  Michele Ceccarelli.  A Finite Markov Random Field approach to fast edge-preserving image recovery , 2007, Image Vis. Comput..

[103]  Juergen Gall,et al.  Class-specific Hough forests for object detection , 2009, CVPR.

[104]  Sebastiano Battiato,et al.  Fuzzy-based motion estimation for video stabilization using SIFT interest points , 2009, Electronic Imaging.

[105]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[106]  Richard A. Brown,et al.  Introduction to random signals and applied kalman filtering (3rd ed , 2012 .