A fast key frame extraction algorithm and an accurate feature matching method for 3D reconstruction from aerial video

In this paper, we present a fast method of key frame selection and an accurate feature point matching method for 3D reconstruction through the control of quadrotor. The quadrotor is controlled to fly at a fixed altitude and the gimbal camera has always been a downward direction. Therefore, the additional information of the quadrotor could be easily added to the 3D reconstruction process. As a result, we find a fast method to select key frames and an optimized matching method through the geometric constraints of the flight path and direction. The method is mainly to improve the efficiency of key frame selection and to get more accurate matching points. The advantage of our approach is to combine the two processes between video capture and video processing by adding flight control to the 3D scene reconstruction method. Therefore, it can use less time to complete the entire reconstruction task. The proposed approach is tested by quadrotor platform. Experiment results show that our method can greatly reduce the key frame selection time and get more matching points at the same accuracy.

[1]  Sang-Hoon Kim,et al.  3D Estimation and Key-Frame Selection for Match Move , 2003 .

[2]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[3]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Fei Dai,et al.  A Novel Approach for Automated Selection of Key Video Frames for 3D Reconstruction of Civil Infrastructure , 2012 .

[5]  Noah Snavely.  Photo Tourism : Exploring image collections in 3D , 2006 .

[6]  Luming Zhang,et al.  Active key frame selection for 3D model reconstruction from crowdsourced geo-tagged videos , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  Simon Fuhrmann,et al.  MVE-A Multiview Reconstruction Environment , 2014 .

[9]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[10]  Andrew W. Fitzgibbon,et al.  Maintaining multiple motion model hypotheses over many views to recover matching and structure , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[11]  Jong-Soo Choi,et al.  Optimal keyframe selection algorithm for three-dimensional reconstruction in uncalibrated multiple images , 2008 .

[12]  Matthew N. Dailey,et al.  Robust Key Frame Extraction for 3D Reconstruction from Video Streams , 2010, VISAPP.

[13]  Shengyong Chen,et al.  Aerial sequential frame decimation for scene reconstruction , 2015, 2015 IEEE International Conference on Information and Automation.

[14]  Pascal Monasse,et al.  Adaptive Structure from Motion with a Contrario Model Estimation , 2012, ACCV.

[15]  Changchang Wu,et al.  SiftGPU : A GPU Implementation of Scale Invariant Feature Transform (SIFT) , 2007 .

[16]  Patricia Ladret,et al.  The blur effect: perception and estimation with a new no-reference perceptual blur metric , 2007, Electronic Imaging.

[17]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[18]  James M. Rehg,et al.  Adaptive Structure from Motion with a Contrario Model Estimation , 2012, ACCV.

[19]  Kuk-Jin Yoon,et al.  Optimal key-frame selection for video-based structure-from-motion , 2011 .