A 3D Reconstruction Pipeline of Urban Drainage Pipes Based on MultiviewImage Matching Using Low-Cost Panoramic Video Cameras

Urban drainage pipe networks have complex spatial contributions, andthey are now facing problems such as damage, defects, and aging. A rapid and high-precision pipe inspection strategy is thekey to ensuring thesustainable development of urban water supply and drainage system. In this paper, a three-dimensional (3D) reconstruction pipeline of urban drainage pipes based on multiview image matching using low-cost panoramic video cameras is proposed, which provides an innovative technical approach for pipe inspection. Firstly, we extracted frames from the panoramic video of the pipes andcorrected the geometric distortion using a spherical reprojection to obtain multiview pipe images. Second, the robust feature matching method using support lines and affine-invariant ratios isintroduced to conduct pipe image matching. Finally, the photogrammetric processing, using structure from motion (SfM) and dense reconstruction, wasintroduced to achieve the 3D modeling of drainage pipes. Several typical drainage pipes and shafts of the real scenes were taken for the 3D reconstruction experiments. Theresults show that our strategy can realize high-precision 3D reconstruction of different types of pipes, which can provide effective technical support for rapid and efficient inspection of urban pipes with broad application prospects in the daily management of sustainable urban drainage systems (SUDSs).

[1]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[2]  Philippe De Smedt,et al.  On introducing an image-based 3D reconstruction method in archaeological excavation practice , 2014 .

[3]  Amir Ali Forough Nassiraei,et al.  Concept and Design of A Fully Autonomous Sewer Pipe Inspection Mobile Robot "KANTARO" , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[4]  Qing Zhu,et al.  Integration of aerial oblique imagery and terrestrial imagery for optimized 3D modeling in urban areas , 2018 .

[5]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[6]  Jianyu Huang,et al.  Rapid 3D Reconstruction for Image Sequence Acquired from UAV Camera , 2018, Sensors.

[7]  F. Cherqui,et al.  The consistency of visual sewer inspection data , 2013 .

[8]  Halina Rubinsztein-Dunlop,et al.  Roadmap on structured light , 2016 .

[9]  Juan Carballini,et al.  Using synthetic aperture sonar as an effective tool for pipeline inspection survey projects , 2015, 2015 IEEE/OES Acoustics in Underwater Geosciences Symposium (RIO Acoustics).

[10]  Franz S. Hover,et al.  Underwater inspection using sonar-based volumetric submaps , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Zheng Liu,et al.  The use of laser range finder on a robotic platform for pipe inspection , 2012 .

[12]  T SilvaClaudio,et al.  A Survey of Surface Reconstruction from Point Clouds , 2017 .

[13]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[14]  Yuxiang Sun,et al.  Improving RGB-D SLAM in dynamic environments: A motion removal approach , 2017, Robotics Auton. Syst..

[15]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Kamal Jain,et al.  Image based 3D city modeling : Comparative study , 2014 .

[17]  Jianhua Liu,et al.  A 3D reconstruction method for pipeline inspection based on multi-vision , 2017 .

[18]  Wei Wu,et al.  Rapid Delaunay triangulation for randomly distributed point cloud data using adaptive Hilbert curve , 2016, Comput. Graph..

[19]  Zhaolin Chen,et al.  3D Texture Mapping in Multi-view Reconstruction , 2012, ISVC.

[20]  G. Kuschk,et al.  Model-free dense stereo reconstruction for creating realistic 3D city models , 2013, Joint Urban Remote Sensing Event 2013.

[21]  Diego González-Aguilera,et al.  GRAPHOS – open‐source software for photogrammetric applications , 2018 .

[22]  Guirong Liu,et al.  Smoothed Finite Element Methods (S-FEM): An Overview and Recent Developments , 2018 .

[23]  Roman Osorio,et al.  Towards 3D Pipe Reconstruction Employing Affine Transformations from Video Information , 2018, IEEE Latin America Transactions.

[24]  Nikolay Strigul,et al.  Embedded, real-time UAV control for improved, image-based 3D scene reconstruction , 2016 .

[25]  Ronen Basri,et al.  A Survey on Structure from Motion , 2017, ArXiv.

[26]  Cettina Santagati,et al.  IMAGE-BASED MODELING TECHNIQUES FOR ARCHITECTURAL HERITAGE 3D DIGITALIZATION: LIMITS AND POTENTIALITIES , 2013 .

[27]  Seungyong Lee,et al.  Texture map generation for 3D reconstructed scenes , 2016, The Visual Computer.

[28]  Martin Byröd,et al.  Conjugate Gradient Bundle Adjustment , 2010, ECCV.

[29]  Tomoaki Mashimo,et al.  Development of spherical ultrasonic motor as a camera actuator for pipe inspection robot , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[30]  Dmitry Chetverikov,et al.  Image-guided ToF depth upsampling: a survey , 2017, Machine Vision and Applications.

[31]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[32]  Hui Zhang,et al.  Efficient 3D Room Shape Recovery from a Single Panorama , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Weiping Cheng,et al.  Study on Pipe Burst Detection Frame Based on Water Distribution Model and Monitoring System , 2019, Water.

[34]  Ming-Der Yang,et al.  Morphological segmentation based on edge detection for sewer pipe defects on CCTV images , 2011, Expert Syst. Appl..

[35]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[36]  Bing-Yu Chen,et al.  High‐resolution 360 Video Foveated Stitching for Real‐time VR , 2017, Comput. Graph. Forum.

[37]  Wei Zhang,et al.  A Unified Framework for Street-View Panorama Stitching , 2016, Sensors.

[38]  Dieter Fritsch,et al.  IMAGE ACQUISITION AND MODEL SELECTION FOR MULTI-VIEW STEREO , 2013 .

[39]  Qingwu Hu,et al.  Robust feature matching via support-line voting and affine-invariant ratios , 2017 .

[40]  Pierre Alliez,et al.  A Survey of Surface Reconstruction from Point Clouds , 2017, Comput. Graph. Forum.

[41]  V Sousa,et al.  Comparison of the inspector and rating protocol uncertainty influence in the condition rating of sewers. , 2014, Water science and technology : a journal of the International Association on Water Pollution Research.

[42]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[43]  Andreas Lanitis,et al.  Image Based 3D Face Reconstruction: a Survey , 2009, Int. J. Image Graph..

[44]  V. A. Knyaz.  Image-Based 3d Reconstruction and Analysis for Orthodontia , 2012 .

[45]  N. Stanic,et al.  A technology for sewer pipe inspection (Part 2): Experimental assessment of a new laser profiler for sewer defect detection and quantification , 2017 .

[46]  Mariette Yvinec,et al.  A surface reconstruction method for in-detail underwater 3D optical mapping , 2015, Int. J. Robotics Res..

[47]  Richard Szeliski,et al.  Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment , 2000, International Journal of Computer Vision.

[48]  Michael Pedrick,et al.  Ultrasonic signal processing methods for detection of defects in concrete pipes , 2012 .

[49]  Olga Sorkine-Hornung,et al.  Point Cloud Noise and Outlier Removal for Image-Based 3D Reconstruction , 2016, 2016 Fourth International Conference on 3D Vision (3DV).

[50]  Eva Pietroni,et al.  Livia's Villa Reloaded: An example of re-use and update of a pre-existing Virtual Museum, following a novel approach in storytelling inside virtual reality environments , 2015, 2015 Digital Heritage.

[51]  Stephan Nebiker,et al.  COMBINING PUBLIC DOMAIN AND PROFESSIONAL PANORAMIC IMAGERY FOR THE ACCURATE AND DENSE 3D RECONSTRUCTION OF THE DESTROYED BEL TEMPLE IN PALMYRA , 2016 .

[53]  Richard Szeliski,et al.  Bundle Adjustment in the Large , 2010, ECCV.

[54]  Joong Hoon Kim,et al.  Robust meter network for water distribution pipe burst detection , 2017 .

[55]  Fabio Bruno,et al.  From 3D reconstruction to virtual reality: A complete methodology for digital archaeological exhibition , 2010 .

[56]  Han-Pang Huang,et al.  Development and Fuzzy Control of a Pipe Inspection Robot , 2010, IEEE Transactions on Industrial Electronics.

[57]  C. Nardinocchi,et al.  Web Spherical Panorama for Cultural Heritage 3D Modeling , 2017 .

[58]  Yi-Ping Chen,et al.  Image-based 3D scene reconstruction and exploration in augmented reality , 2013 .

[59]  Changmin Kim,et al.  3D reconstruction of as-built industrial instrumentation models from laser-scan data and a 3D CAD database based on prior knowledge , 2015 .