Where is photogrammetry heading to? State of the art and trends

The objective of this paper is to highlight current trends in photogrammetry, trying to foresee where they will lead the discipline in the next years. To this aim, first some remarks on the challenges brought to photogrammetry by other sensors and a brief historical survey of some research topics, where an increasing convergence between photogrammetry and computer vision is apparent, will be presented. Then, a necessarily concise review of the advances in automation in three basic photogrammetric tasks (namely image orientation, surface reconstruction and object restitution) will be illustrated. The purpose of the review is to highlight how the fruitful dialog between photogrammetry and computer vision led to today’s achievements and to point out what kind of approaches seem to be winning in the search for viable and robust solutions in the automation of processes. Finally, the conclusions will look at this convergence in the perspective of academic career.

[1]  H. M. Beckwith AUTOMATIC MAP COMPILATION SYSTEM , 1962 .

[2]  G. Piazzesi Photogrammetry with the scanning electron microscope , 1973 .

[3]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[4]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[5]  Madhav Sadashiv Golwalkar,et al.  The integral approach , 1979 .

[6]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[7]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[8]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  A. Gruen ADAPTIVE LEAST SQUARES CORRELATION: A POWERFUL IMAGE MATCHING TECHNIQUE , 1985 .

[10]  David G. Lowe,et al.  Three-Dimensional Object Recognition from Single Two-Dimensional Images , 1987, Artif. Intell..

[11]  Ramakant Nevatia,et al.  Using Perceptual Organization to Extract 3-D Structures , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  John C. Curlander,et al.  Synthetic Aperture Radar: Systems and Signal Processing , 1991 .

[13]  Joseph L. Mundy,et al.  Relationship between photogrammmetry and computer vision , 1993, Defense, Security, and Sensing.

[14]  Takeo Kanade,et al.  A Multiple-Baseline Stereo , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Tuan Dang,et al.  Applying perceptual grouping and surface models to the detection and stereo reconstruction of buildings in aerial imagery , 1994, Other Conferences.

[16]  E. Baltsavias,et al.  Use of DTMs/DSMs and orthoimages to support building extraction , 1995 .

[17]  E. Baltsavias,et al.  Automatic Extraction of Man-Made Objects from Aerial and Space Images (II) , 1995 .

[18]  Rachid Deriche,et al.  A Robust Technique for Matching two Uncalibrated Images Through the Recovery of the Unknown Epipolar Geometry , 1995, Artif. Intell..

[19]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[20]  Wolfgang Förstner,et al.  Towards automatic building extraction from high-resolution digital elevation models , 1995 .

[21]  Pascal Fua,et al.  Taking Advantage of Image-Based and Geometry-Based Constraints to Recover 3-D Surfaces , 1996, Comput. Vis. Image Underst..

[22]  E. Baltsavias,et al.  DSM generation with the Leica/Helava DPW 770 and VirtuoZo digital photogrammetric systems , 1996 .

[23]  Toni Schenk Towards automatic aerial triangulation , 1997 .

[24]  Christian Heipke,et al.  Automation of interior, relative, and absolute orientation , 1997 .

[25]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[27]  B. Bay,et al.  Digital volume correlation: Three-dimensional strain mapping using X-ray tomography , 1999 .

[28]  Fuk K. Li,et al.  Synthetic aperture radar interferometry , 2000, Proceedings of the IEEE.

[29]  David E. Smith,et al.  Mars Orbiter Laser Altimeter: Experiment summary after the first year of global mapping of Mars , 2001 .

[30]  Azriel Rosenfeld,et al.  From Image Analysis to Computer Vision: An Annotated Bibliography, 1955-1979 , 2001, Comput. Vis. Image Underst..

[31]  G. Forlani,et al.  Building extraction from LIDAR data , 2001, IEEE/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas (Cat. No.01EX482).

[32]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[33]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[34]  Xavier Armangué,et al.  Overall view regarding fundamental matrix estimation , 2003, Image Vis. Comput..

[35]  Uwe Hampel,et al.  APPLICATION OF DIGITAL PHOTOGRAMMETRY FOR MEASURING DEFORMATION AND CRACKS DURING LOAD TESTS IN CIVIL ENGINEERING MATERIAL TESTING , 2003 .

[36]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[37]  Philip H. S. Torr,et al.  Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting , 2002, International Journal of Computer Vision.

[38]  M. Scaioni,et al.  A SOLUTION FOR THE GENERAL CASE OF THR THREE IMAGE ORIENTATION , 2004 .

[39]  G. Forlani,et al.  APPLICATION OF DIGITAL PHOTOGRAMMETRY IN GEOTECHNICS , 2004 .

[40]  R. Sukthankar,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[41]  Thierry Toutin,et al.  Comparison of stereo-extracted DTM from different high-resolution sensors: SPOT-5, EROS-a, IKONOS-II, and QuickBird , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[42]  Richard Szeliski,et al.  Stereo Matching with Nonlinear Diffusion , 1998, International Journal of Computer Vision.

[43]  Yan Ke,et al.  PCA-SIFT: a more distinctive representation for local image descriptors , 2004, CVPR 2004.

[44]  Olivier D. Faugeras,et al.  The fundamental matrix: Theory, algorithms, and stability analysis , 2004, International Journal of Computer Vision.

[45]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1999, International Journal of Computer Vision.

[46]  A. Habib,et al.  INTEGRATION OF LIDAR AND PHOTOGRAMMETRY FOR CLOSE RANGE APPLICATIONS , 2004 .

[47]  N. Pfeifer,et al.  AUTOMATIC RECONSTRUCTION OF SINGLE TREES FROM TERRESTRIAL LASER SCANNER DATA , 2004 .

[48]  A. Basilevsky,et al.  Recent and episodic volcanic and glacial activity on Mars revealed by the High Resolution Stereo Camera , 2004, Nature.

[49]  K. Itten,et al.  LIDAR-based geometric reconstruction of boreal type forest stands at single tree level for forest and wildland fire management , 2004 .

[50]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[51]  Primo Zingaretti,et al.  Complete classification of raw LIDAR data and 3D reconstruction of buildings , 2006, Pattern Analysis and Applications.

[52]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[53]  Martin Sonnenschein,et al.  VIRTOPSY--scientific documentation, reconstruction and animation in forensic: individual and real 3D data based geo-metric approach including optical body/object surface and radiological CT/MRI scanning. , 2005, Journal of forensic sciences.

[54]  Helmut Mayer,et al.  BUILDING FACADE INTERPRETATION FROM IMAGE SEQUENCES , 2005 .

[55]  K. Jacobsen,et al.  Direct Sensor Orientation for Large Scale Mapping—Potential, Problems, Solutions , 2005 .

[56]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  John Trinder,et al.  Using the Dempster-Shafer method for the fusion of LIDAR data and multi-spectral images for building detection , 2005, Inf. Fusion.

[58]  C. Brenner Building reconstruction from images and laser scanning , 2005 .

[59]  F. Ackermann,et al.  DIGITAL IMAGE CORRELATION: PERFORMANCE AND POTENTIAL APPLICATION IN PHOTOGRAMMETRY , 2006 .

[60]  A. Guarnieria,et al.  DIGITAL PHOTOGRAMMETRY AND TLS DATA FUSION APPLIED TO CULTURAL HERITAGE 3D MODELING , 2006 .

[61]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[62]  Fabio Remondino,et al.  Image‐based 3D Modelling: A Review , 2006 .

[63]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[64]  Stuart Robson,et al.  Close Range Photogrammetry: Principles, Methods and Applications , 2006 .

[65]  M. Pierrot-Deseilligny,et al.  A MULTIRESOLUTION AND OPTIMIZATION-BASED IMAGE MATCHING APPROACH : AN APPLICATION TO SURFACE RECONSTRUCTION FROM SPOT 5-HRS STEREO IMAGERY , 2006 .

[66]  N. Haala,et al.  COMBINED FEATURE EXTRACTION FOR FAÇADE RECONSTRUCTION , 2007 .

[67]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[68]  H. Hirschmüller Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[69]  Automatic Stereo Plotting , 2008 .

[70]  Tania Landes,et al.  Comparison methods of terrestrial laser scanning, photogrammetry and tacheometry data for recording of cultural heritage buildings , 2008 .

[71]  A. McEwen,et al.  Ultrahigh resolution topographic mapping of Mars with MRO HiRISE stereo images: Meter‐scale slopes of candidate Phoenix landing sites , 2008 .

[72]  A. McEwen,et al.  Quasi-Periodic Bedding in the Sedimentary Rock Record of Mars , 2008, Science.

[73]  P. Pasquali,et al.  DSM generation and deformation measurement from SAR data , 2008 .

[74]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[75]  Norbert Pfeifer,et al.  A Comprehensive Automated 3D Approach for Building Extraction, Reconstruction, and Regularization from Airborne Laser Scanning Point Clouds , 2008, Sensors.

[76]  Norbert Pfeifer,et al.  Real - time data collection and information generation using airborne sensors , 2008 .

[77]  Jan-Michael Frahm,et al.  Developing visual sensing strategies through next best view planning , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[78]  D. Stead,et al.  Close-range terrestrial digital photogrammetry and terrestrial laser scanning for discontinuity characterization on rock cuts , 2009 .

[79]  Siyka Zlatanova,et al.  Geospatial Information Technology for Emergency Response , 2009 .

[80]  George Vosselman,et al.  Knowledge based reconstruction of building models from terrestrial laser scanning data , 2009 .

[81]  Giovanna Sansoni,et al.  State-of-The-Art and Applications of 3D Imaging Sensors in Industry, Cultural Heritage, Medicine, and Criminal Investigation , 2009, Sensors.

[82]  F. Scholten,et al.  Derivation and V alidation of High-Resolution Digital T errain Models fr om Mars Express HRSC Data , 2009 .

[83]  George Vosselman,et al.  Building Reconstruction by Target Based Graph Matching on Incomplete Laser Data: Analysis and Limitations , 2009, Sensors.

[84]  Reynaldo Roque,et al.  An optical strain measurement system for asphalt mixtures , 2009 .

[85]  Gianfranco Forlani,et al.  Advanced Geostructural Survey Methods Applied to Rock Mass Characterization , 2009 .

[86]  Fulvio Rinaudo,et al.  New integration approach of photogrammetric and LIDAR techninques for architectural surveys , 2009 .

[87]  T. H. Kolbe,et al.  CityGML – OGC Standard for Photogrammetry ? , 2009 .

[88]  Frédéric Bretar,et al.  Full-waveform topographic lidar : State-of-the-art , 2009 .

[89]  M. Jaboyedoff,et al.  Use of LIDAR in landslide investigations: a review , 2012, Natural Hazards.

[90]  P. Zingaretti,et al.  Performance evaluation of automated approaches to building detection in multi-source aerial data , 2010 .

[91]  G. Neukum,et al.  Stratigraphy in the Mawrth Vallis region through OMEGA, HRSC color imagery and DTM , 2010 .

[92]  Fabio Remondino,et al.  Orientation and 3D modelling from markerless terrestrial images: combining accuracy with automation , 2010 .

[93]  M. Downey,et al.  SEMI-GLOBAL MATCHING : AN ALTERNATIVE TO LIDAR FOR DSM GENERATION ? , 2010 .

[94]  T. Pock,et al.  Point Clouds: Lidar versus 3D Vision , 2010 .

[95]  Josiane Zerubia,et al.  Structural Approach for Building Reconstruction from a Single DSM , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[96]  George Vosselman,et al.  Airborne and terrestrial laser scanning , 2011, Int. J. Digit. Earth.

[97]  Norbert Haala,et al.  An update on automatic 3D building reconstruction , 2010 .

[98]  Clive S. Fraser,et al.  Automatic Building Detection Using LIDAR Data and Multispectral Imagery , 2010, 2010 International Conference on Digital Image Computing: Techniques and Applications.

[99]  Hao Zhang,et al.  Automatic reconstruction of tree skeletal structures from point clouds , 2010, SIGGRAPH 2010.

[100]  Dušan Petrovič,et al.  Samodejen zajem in iskanje sprememb v topografskem sloju stavb iz digitalnega modela površja in multispektralnega ortofota , 2011 .

[101]  Kaichang Di,et al.  Rigorous Photogrammetric Processing of HiRISE Stereo Imagery for Mars Topographic Mapping , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[102]  D. Grigillo,et al.  Automatic extraction and building change detection from digital surface model and multispectral orthophoto , 2011 .

[103]  F. Fassi,et al.  Surveying and modelling the main spire of Milan Cathedral using multiple data sources , 2011 .

[104]  Fabio Remondino,et al.  Experiences and achievements in automated image sequence orientation for close-range photogrammetric projects , 2011, Optical Metrology.

[105]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[106]  Uwe Soergel,et al.  Conditional Random Fields for Urban Scene Classification with Full Waveform LiDAR Data , 2011, PIA.

[107]  Fabio Remondino,et al.  UAV PHOTOGRAMMETRY FOR MAPPING AND 3D MODELING - CURRENT STATUS AND FUTURE PERSPECTIVES - , 2012 .

[108]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[109]  G. Forlani,et al.  COMPARISON OF TWO STRUCTURE AND MOTION STRATEGIES , 2012 .

[110]  Florent Lafarge,et al.  Creating Large-Scale City Models from 3D-Point Clouds: A Robust Approach with Hybrid Representation , 2012, International Journal of Computer Vision.

[111]  S. Robson,et al.  Straightforward reconstruction of 3D surfaces and topography with a camera: Accuracy and geoscience application , 2012 .

[112]  N. El-Sheimy,et al.  A NEW OBJECT BASED METHOD FOR AUTOMATED EXTRACTION OF URBAN OBJECTS FROM AIRBORNE SENSORS DATA , 2012 .

[113]  Pedro Arias,et al.  Land-Based Mobile Laser Scanning Systems: a Review , 2012 .

[114]  M. Westoby,et al.  ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications , 2012 .

[115]  A. Gruen Development and Status of Image Matching in Photogrammetry , 2012 .

[116]  C. Heipke,et al.  Context-based urban terrain reconstruction from uav-videos for geoinformation applications , 2012 .

[117]  Karsten Schulz,et al.  CONTEXT-BASED URBAN TERRAIN RECONSTRUCTION FROM IMAGES AND VIDEOS , 2012 .

[118]  Michael James,et al.  Straightforward reconstruction of 3D surfaces or DEMs from photos , 2012 .

[119]  M. Jaboyedoff,et al.  Use of LIDAR in landslide investigations: a review , 2012, Natural Hazards.

[120]  J. Höhle OBLIQUE AERIAL IMAGES AND THEIR USE IN CULTURAL HERITAGE DOCUMENTATION , 2013 .

[121]  Fabio Remondino,et al.  From DSM to 3D building models : a quantitative evaluation , 2013 .

[122]  K. Wenzel,et al.  A comparison of dense matching algorithms for scaled surface reconstruction using stereo camera rigs , 2013 .

[123]  Lutz Plümer,et al.  Model driven reconstruction of roofs from sparse LIDAR point clouds , 2013 .

[124]  Tinne Tuytelaars Wide baseline matching , 2014 .

[125]  J. Brasington,et al.  Modeling the topography of shallow braided rivers using Structure-from-Motion photogrammetry , 2014 .

[126]  Fabio Remondino,et al.  State of the art in high density image matching , 2014 .