Scale Accuracy Evaluation of Image-Based 3D Reconstruction Strategies Using Laser Photogrammetry

Rapid developments in the field of underwater photogrammetry have given scientists the ability to produce accurate 3-dimensional (3D) models which are now increasingly used in the representation and study of local areas of interest. This paper addresses the lack of systematic analysis of 3D reconstruction and navigation fusion strategies, as well as associated error evaluation of models produced at larger scales in GPS-denied environments using a monocular camera (often in deep sea scenarios). Based on our prior work on automatic scale estimation of Structure from Motion (SfM)-based 3D models using laser scalers, an automatic scale accuracy framework is presented. The confidence level for each of the scale error estimates is independently assessed through the propagation of the uncertainties associated with image features and laser spot detections using a Monte Carlo simulation. The number of iterations used in the simulation was validated through the analysis of the final estimate behavior. To facilitate the detection and uncertainty estimation of even greatly attenuated laser beams, an automatic laser spot detection method was developed, with the main novelty of estimating the uncertainties based on the recovered characteristic shapes of laser spots with radially decreasing intensities. The effects of four different reconstruction strategies resulting from the combinations of Incremental/Global SfM, and the a priori and a posteriori use of navigation data were analyzed using two distinct survey scenarios captured during the SUBSAINTES 2017 cruise (doi: 10.17600/17001000). The study demonstrates that surveys with multiple overlaps of nonsequential images result in a nearly identical solution regardless of the strategy (SfM or navigation fusion), while surveys with weakly connected sequentially acquired images are prone to produce broad-scale deformation (doming effect) when navigation is not included in the optimization. Thus the scenarios with complex survey patterns substantially benefit from using multiobjective BA navigation fusion. The errors in models, produced by the most appropriate strategy, were estimated at around 1% in the central parts and always inferior to 5% on the extremities. The effects of combining data from multiple surveys were also evaluated. The introduction of additional vectors in the optimization of multisurvey problems successfully accounted for offset changes present in the underwater USBL-based navigation data, and thus minimize the effect of contradicting navigation priors. Our results also illustrate the importance of collecting a multitude of evaluation data at different locations and moments during the survey. Remote Sens. 2019, 11, 2093; doi:10.3390/rs11182093 www.mdpi.com/journal/remotesensing Remote Sens. 2019, 11, 2093 2 of 32

[1]  O. Pizarro,et al.  Large Area 3-D Reconstructions From Underwater Optical Surveys , 2009, IEEE Journal of Oceanic Engineering.

[2]  Adam J. Mathews,et al.  Visualizing and Quantifying Vineyard Canopy LAI Using an Unmanned Aerial Vehicle (UAV) Collected High Density Structure from Motion Point Cloud , 2013, Remote. Sens..

[3]  Kurt Konolige,et al.  CenSurE: Center Surround Extremas for Realtime Feature Detection and Matching , 2008, ECCV.

[4]  Rafael García,et al.  Automatic Scale Estimation of Structure from Motion based 3D Models using Laser Scalers , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[5]  Rudolph van der Merwe,et al.  The unscented Kalman filter for nonlinear estimation , 2000, Proceedings of the IEEE 2000 Adaptive Systems for Signal Processing, Communications, and Control Symposium (Cat. No.00EX373).

[6]  J. Bythell,et al.  Three-dimensional morphometric measurements of reef corals using underwater photogrammetry techniques , 2001, Coral Reefs.

[7]  Klaus Thoeni,et al.  A COMPARISON OF MULTI-VIEW 3D RECONSTRUCTION OF A ROCK WALL USING SEVERAL CAMERAS AND A LASER SCANNER , 2014 .

[8]  Danilo Schneider,et al.  Analysis of Different Methods for 3D Reconstruction of Natural Surfaces from Parallel‐Axes UAV Images , 2015 .

[9]  Alessandro Capra,et al.  ACCURACY ASSESSMENT OF UNDERWATER PHOTOGRAMMETRIC THREE DIMENSIONAL MODELLING FOR CORAL REEFS , 2016 .

[10]  Stergios I. Roumeliotis,et al.  An Efficient Algebraic Solution to the Perspective-Three-Point Problem , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Stefan B. Williams,et al.  Visual sea-floor mapping from low overlap imagery using bi-objective bundle adjustment and constrained motion , 2012, ICRA 2012.

[12]  Gabriel Oliver-Codina,et al.  Underwater Laser-based Structured Light System for one-shot 3D reconstruction , 2014, IEEE SENSORS 2014 Proceedings.

[13]  Jan-Michael Frahm,et al.  A Vote-and-Verify Strategy for Fast Spatial Verification in Image Retrieval , 2016, ACCV.

[14]  James M. Rehg,et al.  Adaptive Structure from Motion with a Contrario Model Estimation , 2012, ACCV.

[15]  Rafael Garcia,et al.  First direct observation of coseismic slip and seafloor rupture along a submarine normal fault and implications for fault slip history , 2016 .

[16]  Sandy Raimondo,et al.  Estimating 3-dimensional colony surface area of field corals , 2007 .

[17]  S. Cocito,et al.  3-D reconstruction of biological objects using underwater video technique and image processing , 2003 .

[18]  Shuhan Shen,et al.  Accurate Multiple View 3D Reconstruction Using Patch-Based Stereo for Large-Scale Scenes , 2013, IEEE Transactions on Image Processing.

[19]  Benjamin P. Neal,et al.  A quick, easy and non‐intrusive method for underwater volume and surface area evaluation of benthic organisms by 3D computer modelling , 2015 .

[20]  D.M. Kocak,et al.  Underwater imaging and optics: Recent advances , 2008, OCEANS 2008.

[21]  R. Prados,et al.  A Novel Blending Technique for Underwater Gigamosaicing , 2012, IEEE Journal of Oceanic Engineering.

[22]  Tao Yu,et al.  Underwater image enhancement algorithm based on RGB channels histogram equalization , 2018, Other Conferences.

[23]  Hanumant Singh,et al.  Robotic tools for deep water archaeology: Surveying an ancient shipwreck with an autonomous underwater vehicle , 2010, J. Field Robotics.

[24]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[25]  Dirk R. Padfield,et al.  Masked FFT registration , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Reinhard Koch,et al.  3D reconstruction based on underwater video from ROV Kiel 6000 considering underwater imaging conditions , 2009, OCEANS 2009-EUROPE.

[27]  S. Robson,et al.  Mitigating systematic error in topographic models derived from UAV and ground‐based image networks , 2014 .

[28]  Y. Rzhanov,et al.  Coastal and Ocean Mapping Center for Coastal and Ocean Mapping 9-2005 UVSD : Software for Detection of Color Underwater Features , 2017 .

[29]  Asgeir J. Sørensen,et al.  Underwater Photogrammetric Mapping of an Intact Standing Steel Wreck with ROV , 2015 .

[30]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[31]  Mariette Yvinec,et al.  A surface reconstruction method for in-detail underwater 3D optical mapping , 2015, Int. J. Robotics Res..

[32]  Armin Gruen,et al.  MONITORING CORAL GROWTH – THE DICHOTOMY BETWEEN UNDERWATER PHOTOGRAMMETRY AND GEODETIC CONTROL NETWORK , 2018, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[33]  C. Mobley Light and Water: Radiative Transfer in Natural Waters , 1994 .

[34]  Lionel Moisan,et al.  Automatic Homographic Registration of a Pair of Images, with A Contrario Elimination of Outliers , 2012, Image Process. Line.

[35]  C. D'Helft,et al.  Simple methods for interactive 3D modeling, measurements, and digital databases of coral skeletons , 2015 .

[36]  S. M. Jong,et al.  Mapping landslide displacements using Structure from Motion (SfM) and image correlation of multi-temporal UAV photography , 2014 .

[37]  Margaret Kalacska,et al.  Freshwater Fish Habitat Complexity Mapping Using Above and Underwater Structure-From-Motion Photogrammetry , 2018, Remote. Sens..

[38]  Stefan B. Williams,et al.  True Color Correction of Autonomous Underwater Vehicle Imagery , 2016, J. Field Robotics.

[39]  Pere Ridao,et al.  Underwater Multi-Vehicle Trajectory Alignment and Mapping Using Acoustic and Optical Constraints , 2016, Sensors.

[40]  Magnus Oskarsson,et al.  Structure from Motion Estimation with Positional Cues , 2013, SCIA.

[41]  I. K. E. Purnama,et al.  UNDERWATER IMAGE ENHANCEMENT USING ADAPTIVE FILTERING FOR ENHANCED SIFT-BASED IMAGE MATCHING , 2013 .

[42]  Stefan B. Williams,et al.  A simple, fast, and repeatable survey method for underwater visual 3D benthic mapping and monitoring , 2017, Ecology and evolution.

[43]  A. Bartoli,et al.  Bi-Objective Bundle Adjustment With Application to Multi-Sensor SLAM , 2010 .

[44]  Alessandro Capra,et al.  Detecting change in coral reef 3D structure using underwater photogrammetry: critical issues and performance metrics , 2020, Applied Geomatics.

[45]  G. Bianco,et al.  A COMPARISON BETWEEN ACTIVE AND PASSIVE TECHNIQUES FOR UNDERWATER 3D APPLICATIONS , 2012 .

[46]  Peter I. Corke,et al.  Quantifying Multiscale Habitat Structural Complexity: A Cost-Effective Framework for Underwater 3D Modelling , 2016, Remote. Sens..

[47]  Alessandro Capra,et al.  3D RECONSTRUCTION OF AN UNDERWATER ARCHAELOGICAL SITE: COMPARISON BETWEEN LOW COST CAMERAS , 2015 .

[48]  Michael Goesele,et al.  Let There Be Color! Large-Scale Texturing of 3D Reconstructions , 2014, ECCV.

[49]  Gianfranco Forlani,et al.  Quality Assessment of DSMs Produced from UAV Flights Georeferenced with On-Board RTK Positioning , 2018, Remote. Sens..

[50]  D. Green,et al.  Application of Structure‐from‐Motion photogrammetry to river restoration , 2017 .

[51]  Malcolm B. Jones,et al.  ROV Image Scaling with Laser Spot Patterns , 2000 .

[52]  Adrien Bartoli,et al.  Fast Explicit Diffusion for Accelerated Features in Nonlinear Scale Spaces , 2013, BMVC.

[53]  Georgios D. Evangelidis,et al.  Parametric Image Alignment Using Enhanced Correlation Coefficient Maximization , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  A. C. Aitken IV.—On Least Squares and Linear Combination of Observations , 1936 .

[55]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[56]  Jules S. Jaffe,et al.  Computer modeling and the design of optimal underwater imaging systems , 1990 .

[57]  Mark A. Fonstad,et al.  Topographic structure from motion: a new development in photogrammetric measurement , 2013 .

[58]  M. Kalacska,et al.  Structure from motion will revolutionize analyses of tidal wetland landscapes , 2017 .

[59]  J. Lutes,et al.  ACCURACY ASSESSMENT OF DIRECT GEOREFERENCING FOR PHOTOGRAMMETRIC APPLICATIONS ON SMALL UNMANNED AERIAL PLATFORMS , 2016 .

[60]  Pascal Monasse,et al.  Global Fusion of Relative Motions for Robust, Accurate and Scalable Structure from Motion , 2013, ICCV.

[61]  Jordan Mertes,et al.  Rapid, Quantitative Assessment of Submerged Cultural Resource Degradation Using Repeat Video Surveys and Structure from Motion , 2017 .

[62]  Oscar Pizarro,et al.  The Catlin Seaview Survey – kilometre‐scale seascape assessment, and monitoring of coral reef ecosystems , 2014 .

[63]  Tim W. Nattkemper,et al.  DELPHI—fast and adaptive computational laser point detection and visual footprint quantification for arbitrary underwater image collections , 2015, Front. Mar. Sci..

[64]  Hauke Strasdat,et al.  Scale Drift-Aware Large Scale Monocular SLAM , 2010, Robotics: Science and Systems.

[65]  Vincent Raoult,et al.  How Reliable Is Structure from Motion (SfM) over Time and between Observers? A Case Study Using Coral Reef Bommies , 2017, Remote. Sens..

[66]  Pierre-Marie Sarradin,et al.  Victor 6000 : design, utilisation and first improvements , 2003 .

[67]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[68]  Gianfranco Bianco,et al.  A fast enhancing method for non-uniformly illuminated underwater images , 2017, OCEANS 2017 – Anchorage.

[69]  Ben Upcroft,et al.  Towards automated and in-situ, near-real time 3-D reconstruction of coral reef environments , 2011, OCEANS'11 MTS/IEEE KONA.

[70]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[71]  Andrey Soloviev,et al.  Integration of GPS and vision measurements for navigation in GPS challenged environments , 2010, IEEE/ION Position, Location and Navigation Symposium.

[72]  Tomas Pajdla,et al.  Exploiting Visibility Information in Surface Reconstruction to Preserve Weakly Supported Surfaces , 2014, International scholarly research notices.

[73]  J. Brasington,et al.  Modeling the topography of shallow braided rivers using Structure-from-Motion photogrammetry , 2014 .

[74]  Eduard Vidal,et al.  Autonomous Underwater Navigation and Optical Mapping in Unknown Natural Environments , 2016, Sensors.