3D pipe reconstruction employing video information from mobile robots

Abstract The development of a virtual scenario is desirable in many hostile environments such as in mining or sewer pipelines to inspect its current condition and to identify possible failures. Obtaining a virtual image through the reconstruction of the mineshaft or the inner pipe is a valuable tool for assessment and fault detection. The approach presented in this paper uses a fuzzy controlled robot so that the video information can reliably be taken. The robot fuzzy control guarantees its motion in a straight line while taking the images from the pipe. The control algorithm therefore prevents the robot from overturning while moving inside the pipe. Results from our 3D reconstruction approach using the autonomous mobile robot is presented. The proposed method overcomes the limitations of different detectors by using a novel combination of DoG and Harris Laplace. Once the frame selection is made, then the regions with affine transformations are used to find image features as correspondences using image descriptors such as SIFT (Scale-Invariant Feature Transform). The correspondences are required to obtain the orientation and position of the camera and to obtain the scene structure leading to the creation of the 3D virtual environment. The 3D reconstruction method outperformed similar commercial software which highlights its potential.

[1]  Kaspar Althoefer,et al.  State of the art in sensor technologies for sewer inspection , 2002 .

[2]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[3]  Three-dimensional measurement of an inner surface profile using a supercontinuum beam. , 2018, Applied optics.

[4]  Cordelia Schmid,et al.  A Performance Evaluation of Local Descriptors , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Reinhard Klette,et al.  Concise Computer Vision: An Introduction into Theory and Algorithms , 2014 .

[6]  Armando Barreto,et al.  Implementing a Sensor Fusion Algorithm for 3D Orientation Detection with Inertial/Magnetic Sensors , 2015 .

[7]  Luc Van Gool,et al.  Simultaneous Object Recognition and Segmentation from Single or Multiple Model Views , 2006, International Journal of Computer Vision.

[8]  Thomas Martin Deserno,et al.  Feature description with SIFT, SURF, BRIEF, BRISK, or FREAK? A general question answered for bone age assessment , 2016, Comput. Biol. Medicine.

[9]  Tal Hassner,et al.  Dense Correspondences across Scenes and Scales , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Richard M. Everson,et al.  Automated detection of faults in sewers using CCTV image sequences , 2018, Automation in Construction.

[11]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Different Scenes , 2008, ECCV.

[12]  Andrew Zisserman,et al.  Multi-view Matching for Unordered Image Sets, or "How Do I Organize My Holiday Snaps?" , 2002, ECCV.

[13]  Mahmoud R. Halfawy,et al.  Automated defect detection in sewer closed circuit television images using histograms of oriented gradients and support vector machine , 2014 .

[14]  Zheng Liu,et al.  State of the art review of inspection technologies for condition assessment of water pipes , 2013 .

[15]  M. Abdellatif,et al.  Mechatronics Design of an Autonomous Pipe-Inspection Robot , 2018 .

[16]  J. Yen,et al.  Fuzzy Logic: Intelligence, Control, and Information , 1998 .

[17]  Jeroen Langeveld,et al.  A technology for sewer pipe inspection (part 1): Design, calibration, corrections and potential application of a laser profiler , 2017 .

[18]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[19]  Wang Ying,et al.  Pipe Defect Detection and Reconstruction Based on 3D Points Acquired by the Circular Structured Light Vision , 2013 .

[20]  Juho Kannala,et al.  Measuring and modelling sewer pipes from video , 2007, Machine Vision and Applications.

[21]  Cordelia Schmid,et al.  3D Object Modeling and Recognition Using Local Affine-Invariant Image Descriptors and Multi-View Spatial Constraints , 2006, International Journal of Computer Vision.

[22]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[23]  Dulcy M. Abraham,et al.  Automated defect classification in sewer closed circuit television inspections using deep convolutional neural networks , 2018, Automation in Construction.

[24]  Mohamed S. Shehata,et al.  Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images , 2017, ArXiv.

[25]  Marc Edwards,et al.  Developing a Framework for Selecting Condition Assessment Technologies for Water and Wastewater Pipes , 2010 .

[26]  Armando Albertazzi,et al.  Laser Triangulation Profilometer for Inner Surface Inspection of 100 millimeters (4") Nominal Diameter , 2015 .

[27]  Paul F. Boulos.  Optimal Scheduling of Pipe Replacement , 2017 .

[28]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  James H. Garrett,et al.  Automated defect detection for sewer pipeline inspection and condition assessment , 2009 .

[30]  Juho Kannala,et al.  A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  A. Patil,et al.  An adaptive approach for the reconstruction and modeling of as-built 3D pipelines from point clouds , 2017 .

[32]  Kai-Wei Chiang,et al.  An intelligent navigator for seamless INS/GPS integrated land vehicle navigation applications , 2008, Appl. Soft Comput..

[33]  Ting Wu,et al.  Detection of morphology defects in pipeline based on 3D active stereo omnidirectional vision sensor , 2017, IET Image Process..

[34]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[35]  Ce Liu,et al.  Deformable Spatial Pyramid Matching for Fast Dense Correspondences , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.