Orientation and 3D modelling from markerless terrestrial images: combining accuracy with automation

In this paper an automated methodology is presented (i) to orient a set of close‐range images captured with a calibrated camera, and (ii) to extract dense and accurate point clouds starting from the estimated orientation parameters. The whole procedure combines different algorithms and techniques in order to obtain accurate 3D reconstructions in an automatic way. The exterior orientation parameters are estimated using a photogrammetric bundle adjustment with the image correspondences detected using area‐ and feature‐based matching algorithms. Surface measurements are then performed using advanced multi‐image matching techniques based on multiple image primitives. To demonstrate the reliability, precision and robustness of the procedure, several tests on different kinds of free‐form objects are illustrated and discussed in the paper. Three‐dimensional comparisons with range‐based data are also carried out.

[1]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[2]  A. Gruen.  ADAPTIVE LEAST SQUARES CORRELATION: A POWERFUL IMAGE MATCHING TECHNIQUE , 1985 .

[3]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[5]  Peter J. Rousseeuw,et al.  Robust Regression and Outlier Detection , 2005, Wiley Series in Probability and Statistics.

[6]  Sanford Weisberg,et al.  Directions in Robust Statistics and Diagnostics , 1991 .

[7]  Takeo Kanade,et al.  A multiple-baseline stereo , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Philippe Saint-Marc,et al.  Adaptive Smoothing: A General Tool for Early Vision , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Emmanuel P. Baltsavias,et al.  Multiphoto geometrically constrained matching , 1991 .

[10]  Peter J. Huber,et al.  Between Robustness and Diagnostics , 1991 .

[11]  Philip H. S. Torr,et al.  Outlier detection and motion segmentation , 1993, Other Conferences.

[12]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[13]  Haihong Li.  Semi-automatic road extraction from satellite and aerial images , 1997 .

[14]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[16]  Andrew Zisserman,et al.  Robust computation and parametrization of multiple view relations , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[17]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[18]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[19]  H. Handley.  AUTOMATION IN VIDEOGRAMMETRY , 2001 .

[20]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[21]  Nicola D'Apuzzo.  Surface measurement and tracking of human body parts from multi-image video sequences , 2002 .

[22]  Luc Van Gool,et al.  Dense matching of multiple wide-baseline views , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[23]  Xavier Armangué,et al.  Overall view regarding fundamental matrix estimation , 2003, Image Vis. Comput..

[24]  Xiangyang Hao,et al.  ORIENTATION AND AUTO-CALIBRATION OF IMAGE TRIPLETS AND SEQUENCES , 2003 .

[25]  Matthew A. Brown,et al.  Recognising panoramas , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Andrew Jones,et al.  Digital Reunification of the Parthenon and its Sculptures , 2003, VAST.

[28]  G. Büyüksalih,et al.  Practical experiences with automatic aerial triangulation using different software packages , 2003 .

[29]  Reinhard Koch,et al.  Self-Calibration and Metric Reconstruction Inspite of Varying and Unknown Intrinsic Camera Parameters , 1999, International Journal of Computer Vision.

[30]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Philip H. S. Torr,et al.  Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting , 2002, International Journal of Computer Vision.

[32]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[33]  R. Sukthankar,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[34]  Zhang Li,et al.  AUTOMATIC DSM GENERATION FROM LINEAR ARRAY IMAGERY DATA , 2004 .

[35]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[36]  D. Nistér.  Automatic passive recovery of 3D from images and video , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[37]  François Blais.  Review of 20 years of range sensor development , 2004, J. Electronic Imaging.

[38]  Sabry F. El-Hakim,et al.  Detailed 3D reconstruction of large-scale heritage sites with integrated techniques , 2004, IEEE Computer Graphics and Applications.

[39]  Roberto Cipolla,et al.  Modelling and Interpretation of Architecture from Several Images , 2004, International Journal of Computer Vision.

[40]  Fabio Remondino,et al.  Photogrammetry for geological applications: automatic retrieval of discontinuity orientation in rock slopes , 2005 .

[41]  R. Reulke,et al.  Remote Sensing and Spatial Information Sciences , 2005 .

[42]  Roberto Cipolla,et al.  Multi-view stereo via volumetric graph-cuts , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[43]  Nicolas Paparoditis,et al.  3D BUILDING FACADE RECONSTRUCTION UNDER MESH FORM FROM MULTIPLE WIDE ANGLE VIEWS , 2005 .

[44]  Cordelia Schmid,et al.  A performance evaluation of local descriptors , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Li Zhang.  Automatic Digital Surface Model (DSM) generation from linear array images , 2005 .

[46]  T. Ohdake,et al.  3D MODELING OF HIGH RELIEF SCULPTURE USING IMAGE BASED INTEGRATED MEASUREMENT SYSTEM , 2005 .

[47]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[48]  Fabio Remondino,et al.  Image‐based 3D Modelling: A Review , 2006 .

[49]  S. I. Granshaw,et al.  BUNDLE ADJUSTMENT METHODS IN ENGINEERING PHOTOGRAMMETRY , 2006 .

[50]  Li Zhang,et al.  SURFACE RECONSTRUCTION ALGORITHMS FOR DETAILED CLOSE-RANGE OBJECT MODELING , 2006 .

[51]  C. Fraser,et al.  Automated metric calibration of colour digital cameras , 2006 .

[52]  Luc Van Gool,et al.  Procedural modeling of buildings , 2006, SIGGRAPH 2006.

[53]  Fabio Remondino.  DETECTORS AND DESCRIPTORS FOR PHOTOGRAMMETRIC APPLICATIONS , 2006 .

[54]  Livio De Luca,et al.  Reverse engineering of architectural buildings based on a hybrid modeling approach , 2006, Comput. Graph..

[55]  Michael Goesele,et al.  Multi-View Stereo Revisited , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[56]  Fabio Remondino,et al.  OVERVIEW AND EXPERIENCES IN AUTOMATED MARKERLESS IMAGE ORIENTATION , 2006 .

[57]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[58]  C. Fraser,et al.  Digital camera calibration methods: Considerations and comparisons , 2006 .

[59]  E. H. Thompson.  A RATIONAL ALGEBRAIC FORMULATION OF THE PROBLEM OF RELATIVE ORIENTATION , 2006 .

[60]  T. Läbe.  AUTOMATIC RELATIVE ORIENTATION OF IMAGES , 2006 .

[61]  Achim J. Lilienthal,et al.  SIFT, SURF and Seasons: Long-term Outdoor Localization Using Local Features , 2007, EMCR.

[62]  Ioannis Stamos,et al.  Integrating Automated Range Registration with Multiview Geometry for the Photorealistic Modeling of Large-Scale Scenes , 2008, International Journal of Computer Vision.

[63]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[64]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[65]  P. Litkey,et al.  INTEGRATION OF LASER SCANNING AND PHOTOGRAMMETRY , 2007 .

[66]  Li Zhang,et al.  Turning images into 3-D models , 2008, IEEE Signal Processing Magazine.

[67]  Paolo Cignoni,et al.  Sampled 3D models for CH applications: A viable and enabling new medium or just a technological exercise? , 2008, JOCCH.

[68]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[69]  Armin Gruen,et al.  Turning Images into 3-D Models ( Developments and performance analysis of image matching for detailed surface reconstruction of heritage objects ) , 2008 .

[70]  David G. Lowe,et al.  Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration , 2009, VISAPP.

[71]  L. Gonzo,et al.  3 D VIRTUAL RECONSTRUCTION AND VISUALIZATION OF COMPLEX ARCHITECTURES – THE “ 3 D-ARCH ” PROJECT , 2009 .

[72]  Luigi Barazzetti,et al.  Generazione di un DTM con integrazione di tecniche laser scanning e fotogrammetriche terrestri , 2009 .

[73]  Gabriele Guidi,et al.  A Multi-Resolution Methodology for the 3D Modeling of Large and Complex Archeological Areas , 2009 .

[74]  Richard Szeliski,et al.  Building Rome in a day , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[75]  David S. Ebert,et al.  Texturing & modeling : a procedural approach : 日本語版 , 2009 .

[76]  Fabio Remondino,et al.  3D Virtual reconstruction and visualization of complex architectures: The 3D-ARCH project , 2009 .

[77]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[78]  C. Fraser,et al.  Interest operators for feature‐based matching in close range photogrammetry , 2010 .

[79]  David S. Ebert,et al.  Texturing and Modeling, Third Edition: A Procedural Approach (The Morgan Kaufmann Series in Computer Graphics) , 2011 .

[80]  Mathias J.P.M. Lemmens,et al.  A SURVEY ON STEREO MATCHING TECHNIQUES , 2012 .