Condition assessment of ship structure using robot assisted 3D-reconstruction

ABSTRACT Ships condition is assessed regularly to maintain safety. Traditionally, structural integrity assessment is performed by surveyors, requiring complex and time-consuming operations to accessany ship space. Imagery based, three-dimensional (3D) reconstruction of structures is a new area obtaining considerable interest. It can provide low-cost, less disruptive and safer inspection approach. In this study, alternative technologies to generate 3D models, based on photos, are explored. The aim is to highlight how human made ship survey can be improved using robotics technology. A procedure for 3D reconstruction combining photogrammetry/videogrammetry and computer vision techniques is developed, providing an alternative to ease vessel inspections. Moreover, effect of pre-processing image datasets, aimed at improving the performance of 3D reconstruction, is investigated. An efficient image pre-processing pipeline based on computer vision algorithms for colour enhancement, shadow removal and image blurriness is presented. This study can help in effective and reliable decision-making process, due to its user-friendliness and cost effectiveness, mainly for cargo holds requiring frequent assessment because of cargo operations induced damages.

[1]  Antonio M. López,et al.  Road Detection Based on Illuminant Invariance , 2011, IEEE Transactions on Intelligent Transportation Systems.

[2]  Xiaohui Zhou,et al.  Survey on path and view planning for UAVs , 2020, Virtual Real. Intell. Hardw..

[3]  Sabine Knapp,et al.  Ship Inspection Strategies: Effects on Maritime Safety and Environmental Protection , 2010 .

[4]  Michael Goesele,et al.  Scene Reconstruction and Visualization From Community Photo Collections , 2010, Proceedings of the IEEE.

[5]  Xiang Zhu,et al.  Deconvolving PSFs for a Better Motion Deblurring Using Multiple Images , 2012, ECCV.

[6]  Cheng Lu,et al.  On the removal of shadows from images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Marco Gaiotti,et al.  Assessment of ship robotic inspections , 2020, 2020 25th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA).

[8]  Francis Guédé.  Risk-based structural integrity management for offshore jacket platforms , 2018 .

[9]  Veronica Teichrieb,et al.  Challenges in 3D Reconstruction from Images for Difficult Large-Scale Objects: A Study on the Modeling of Electrical Substations , 2012, 2012 14th Symposium on Virtual and Augmented Reality.

[10]  Ran Yan,et al.  Ship Inspection by Port State Control—Review of Current Research , 2019, Smart Innovation, Systems and Technologies.

[11]  Marco Gaiotti,et al.  Recent developments in remote inspections of ship structures , 2020 .

[12]  Ahmad Y. Javaid,et al.  A comprehensive micro unmanned aerial vehicle (UAV/Drone) forensic framework , 2019, Digit. Investig..

[13]  Ali Khaloo,et al.  Utilizing UAV and 3D Computer Vision for Visual Inspection of a Large Gravity Dam , 2018, Front. Built Environ..

[14]  Zhiliang Ma,et al.  A review of 3D reconstruction techniques in civil engineering and their applications , 2018, Adv. Eng. Informatics.

[15]  K. Naga Sundari,et al.  REMOTE SENSING UAV/DRONE TECHNOLOGY AS A TOOL FOR URBAN DEVELOPMENT MEASURES IN APCRDA , 2019, The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences.

[16]  Yanning Zhang,et al.  Multi-image Blind Deblurring Using a Coupled Adaptive Sparse Prior , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[17]  Brendt Wohlberg,et al.  Blind Image Deconvolution Motion Blur Estimation , 2006 .

[18]  Lili Liu,et al.  An Integrated Coating Inspection System for Marine and Offshore Corrosion Management , 2018, 2018 15th International Conference on Control, Automation, Robotics and Vision (ICARCV).

[19]  Gerasimos Theotokatos,et al.  Inspection capabilities for enhanced ship safety , 2015 .

[20]  Paul A. Beardsley,et al.  3D Model Acquisition from Extended Image Sequences , 1996, ECCV.

[21]  Qingwu Hu,et al.  A 3D Reconstruction Pipeline of Urban Drainage Pipes Based on MultiviewImage Matching Using Low-Cost Panoramic Video Cameras , 2019, Water.

[22]  Xin Wang,et al.  Algorithm of Crane Selection for Heavy Lifts , 2011 .

[23]  Andrew Owens,et al.  Discrete-continuous optimization for large-scale structure from motion , 2011, CVPR 2011.

[24]  Franz S. Hover,et al.  Advanced perception, navigation and planning for autonomous in-water ship hull inspection , 2012, Int. J. Robotics Res..

[25]  Markus Eich,et al.  Design and Control of MIRA: A Lightweight Climbing Robot for Ship Inspection , 2015 .

[26]  Enrico,et al.  Kit of Self-Moving Units for Automatic Inspection of Marine Structures and Plants： Low Cost Unit for Inspection of Holds , 2016 .

[27]  Zhisen Yang,et al.  Realising advanced risk-based port state control inspection using data-driven Bayesian networks , 2018 .

[28]  Umberto Papa,et al.  Preliminary Design of an Unmanned Aircraft System for Aircraft General Visual Inspection , 2018, Electronics.

[29]  Ján Kaňuk,et al.  Combined Use of Terrestrial Laser Scanning and UAV Photogrammetry in Mapping Alpine Terrain , 2019, Remote. Sens..

[30]  Zhiliang Ma,et al.  A review of 3 D reconstruction techniques in civil engineering and their applications , 2018 .

[31]  Lennart Elfgren,et al.  3D reconstruction of existing concrete bridges using optical methods , 2019, Structure and Infrastructure Engineering.

[32]  Sylvain Paris,et al.  Blur kernel estimation using the radon transform , 2011, CVPR 2011.

[33]  Pascal Monasse,et al.  Adaptive Structure from Motion with a Contrario Model Estimation , 2012, ACCV.

[34]  Erik Champion,et al.  To 3D or Not 3D: Choosing a Photogrammetry Workflow for Cultural Heritage Groups , 2019, Heritage.

[35]  Lars Linsen,et al.  Interactive History-Based Vessel Movement Prediction , 2019, IEEE Intelligent Systems.

[36]  Pawel Burdziakowski,et al.  Combined Close Range Photogrammetry and Terrestrial Laser Scanning for Ship Hull Modelling , 2019, Geosciences.

[37]  Tobias Höllerer,et al.  Optimizing the Viewing Graph for Structure-from-Motion , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[38]  Li Zhang,et al.  Turning images into 3-D models , 2008, IEEE Signal Processing Magazine.

[39]  Akio Namiki,et al.  Autonomous Target Tracking of UAV Using High-Speed Visual Feedback , 2019 .

[40]  O. Sass,et al.  Combining airborne and terrestrial laser scanning for quantifying erosion and deposition by a debris flow event , 2012 .

[41]  Fernanda Leite,et al.  Evaluation of accuracy of as-built 3D modeling from photos taken by handheld digital cameras , 2012 .

[42]  Salvador Senent,et al.  Structure from Motion photogrammetry to characterize underground rock masses: Experiences from two real tunnels , 2019, Tunnelling and Underground Space Technology.

[43]  T. S. Akinfiev,et al.  Nondestructive testing of the state of a ship’s hull with an underwater robot , 2008 .

[44]  Graham D. Finlayson,et al.  Hamiltonian Path based Shadow Removal , 2005, BMVC.

[45]  Junaed Sattar,et al.  Toward a Generic Diver-Following Algorithm: Balancing Robustness and Efficiency in Deep Visual Detection , 2018, IEEE Robotics and Automation Letters.

[46]  Victor M. Becerra,et al.  Autonomous Control of Unmanned Aerial Vehicles , 2019, Electronics.

[47]  Nicholas C. Coops,et al.  Digital Aerial Photogrammetry for Updating Area-Based Forest Inventories: A Review of Opportunities, Challenges, and Future Directions , 2019, Current Forestry Reports.

[48]  David Bienvenido-Huertas,et al.  Validation of Close-Range Photogrammetry for Architectural and Archaeological Heritage: Analysis of Point Density and 3D Mesh Geometry , 2020, Remote. Sens..

[49]  Krzysztof Sterenczak,et al.  Digital Aerial Photogrammetry (DAP) and Airborne Laser Scanning (ALS) as Sources of Information about Tree Height: Comparisons of the Accuracy of Remote Sensing Methods for Tree Height Estimation , 2020, Remote. Sens..

[50]  Hyoung Il Son,et al.  Multiple UAV Systems for Agricultural Applications: Control, Implementation, and Evaluation , 2018, Electronics.

[51]  T. Gaggero,et al.  Robotic inspection of ships: inherent challenges and assessment of their effectiveness , 2020, Ships and Offshore Structures.

[52]  Airong Chen,et al.  Three-Dimensional Reconstruction of Structural Surface Model of Heritage Bridges Using UAV-Based Photogrammetric Point Clouds , 2019, Remote. Sens..

[53]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[54]  Long Quan,et al.  Relative 3D Reconstruction Using Multiple Uncalibrated Images , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Ioannis Brilakis,et al.  Comparison of Optical Sensor-Based Spatial Data Collection Techniques for Civil Infrastructure Modeling , 2009 .

[56]  Mohan Rajesh Elara,et al.  Visual Inspection of the Aircraft Surface Using a Teleoperated Reconfigurable Climbing Robot and Enhanced Deep Learning Technique , 2019, International Journal of Aerospace Engineering.

[57]  Andrea Fusiello,et al.  Improving the efficiency of hierarchical structure-and-motion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.