Aerial Implicit 3D Video Stabilization Using Epipolar Geometry Constraint

We present an accurate video stabilization method on aerial videos using the epipolar geometry constraint. Most previous methods used 2D homography for stabilization, but failed to overcome the parallax problem. In this work, we propose to use dense correspondences for stabilization and the epipolar constraint to deal with the parallax effect. We start by estimating the dense correspondences between two frames. The dense correspondences are then used to estimate the epipolar geometry. The epipolar geometry has an implicit 3D constraint that can be used to improve the dense correspondences and handle the parallax. We evaluate our method on a real-life database containing three aerial image sequences. We also compare our method with the dominant 2D method for aerial video stabilization. The quantitative result demonstrates the effectiveness of our approach.

[1]  Horst Bischof,et al.  Motion estimation with non-local total variation regularization , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Michael Bosse,et al.  Non-metric image-based rendering for video stabilization , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Rama Chellappa,et al.  Evaluation of image stabilization algorithms , 1998, Proceedings of the 1998 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP '98 (Cat. No.98CH36181).

[4]  Harry Shum,et al.  Full-frame video stabilization with motion inpainting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Andrew J. Davison,et al.  Live dense reconstruction with a single moving camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Rachid Deriche,et al.  Symmetrical Dense Optical Flow Estimation with Occlusions Detection , 2002, ECCV.

[7]  Jongmoo Choi,et al.  Activity recognition in wide aerial video surveillance using entity relationship models , 2012, SIGSPATIAL/GIS.

[8]  Haroon Idrees,et al.  Detection and Tracking of Large Number of Targets in Wide Area Surveillance , 2010, ECCV.

[9]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[10]  Irfan A. Essa,et al.  Auto-directed video stabilization with robust L1 optimal camera paths , 2011, CVPR 2011.

[11]  Michael Gleicher,et al.  Re-cinematography: improving the camera dynamics of casual video , 2007, ACM Multimedia.

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  Harpreet S. Sawhney,et al.  Vehicle detection and tracking in wide field-of-view aerial video , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Grace Vesom,et al.  Pixel-wise motion detection in persistent aerial video surveillance , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[15]  Bing-Yu Chen,et al.  Capturing Intention‐based Full‐Frame Video Stabilization , 2008, Comput. Graph. Forum.

[16]  Joachim Weickert,et al.  Dense versus Sparse Approaches for Estimating the Fundamental Matrix , 2011, International Journal of Computer Vision.

[17]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[18]  Gérard G. Medioni,et al.  Using 3D scene structure to improve tracking , 2011, CVPR 2011.

[19]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[20]  Michael Gleicher,et al.  Subspace video stabilization , 2011, TOGS.

[21]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[22]  Hailin Jin,et al.  Light field video stabilization , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[23]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..