Optimization of incremental structure from motion combining a random k-d forest and pHash for unordered images in a complex scene

Abstract. On the basis of today’s popular virtual reality and scientific visualization, three-dimensional (3-D) reconstruction is widely used in disaster relief, virtual shopping, reconstruction of cultural relics, etc. In the traditional incremental structure from motion (incremental SFM) method, the time cost of the matching is one of the main factors restricting the popularization of this method. To make the whole matching process more efficient, we propose a preprocessing method before the matching process: (1) we first construct a random k-d forest with the large-scale scale-invariant feature transform features in the images and combine this with the pHash method to obtain a value of relatedness, (2) we then construct a connected weighted graph based on the relatedness value, and (3) we finally obtain a planned sequence of adding images according to the principle of the minimum spanning tree. On this basis, we attempt to thin the minimum spanning tree to reduce the number of matchings and ensure that the images are well distributed. The experimental results show a great reduction in the number of matchings with enough object points, with only a small influence on the inner stability, which proves that this method can quickly and reliably improve the efficiency of the SFM method with unordered multiview images in complex scenes.

[1]  Andrea Fusiello,et al.  Structure-and-motion pipeline on a hierarchical cluster tree , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[2]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[3]  C. Heipke,et al.  An efficient method to detect mutual overlap of a large set of unordered images for structure-from-motion , 2017 .

[4]  Xin Wang,et al.  Fast method of constructing image correlations to build a free network based on image multivocabulary trees , 2015, J. Electronic Imaging.

[5]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[6]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[7]  Javier-Flavio Vigueras,et al.  Phase Correlation Based Image Alignment with Subpixel Accuracy , 2012, MICAI.

[8]  D. Lowe,et al.  Fast Matching of Binary Features , 2012, 2012 Ninth Conference on Computer and Robot Vision.

[9]  Ton Kalker,et al.  A Highly Robust Audio Fingerprinting System , 2002, ISMIR.

[10]  C. D. Kuglin,et al.  The phase correlation image alignment method , 1975 .

[11]  Ramarathnam Venkatesan,et al.  Robust perceptual image hashing via matrix invariants , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[12]  Ramarathnam Venkatesan,et al.  A Perceptual Audio Hashing Algorithm: A Tool for Robust Audio Identification and Information Hiding , 2001, Information Hiding.

[13]  David G. Lowe,et al.  Scalable Nearest Neighbor Algorithms for High Dimensional Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  James M. Rehg,et al.  Adaptive Structure from Motion with a Contrario Model Estimation , 2012, ACCV.

[15]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[16]  He Chen,et al.  A coarse-to-fine image registration method based on visual attention model , 2014, Science China Information Sciences.

[17]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[18]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[19]  P. J. Narayanan,et al.  Multistage SFM: Revisiting Incremental Structure from Motion , 2014, 2014 2nd International Conference on 3D Vision.

[20]  Kenneth Moreland,et al.  The FFT on a GPU , 2003, HWWS '03.

[21]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[22]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[23]  Richard I. Hartley,et al.  Optimised KD-trees for fast image descriptor matching , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[25]  Chaomei Chen,et al.  Visualizing evolving networks: minimum spanning trees versus pathfinder networks , 2003, IEEE Symposium on Information Visualization 2003 (IEEE Cat. No.03TH8714).