Dense 3D Reconstruction for Visual Tunnel Inspection using Unmanned Aerial Vehicle

Advances in Unmanned Aerial Vehicle (UAV) opens venues for application such as tunnel inspection. Owing to its versatility to fly inside the tunnels, it can quickly identify defects and potential problems related to safety. However, long tunnels, especially with repetitive or uniform structures pose a significant problem for UAV navigation. Furthermore, post-processing visual data from the camera mounted on the UAV is required to generate useful information for the inspection task. In this work, we design a UAV with a single rotating camera to accomplish the task. Compared to other platforms, our solution can fit the stringent requirement for tunnel inspection, in terms of battery life, size and weight. While the current state-of-the-art can estimate camera pose and 3D geometry from a sequence of images, they assume large overlap, small rotational motion, and many distinct matching points between images. These assumptions severely limit their effectiveness in tunnel-like scenarios where the camera has erratic or large rotational motion, such as the one mounted on the UAV. This paper presents a novel solution which exploits Structure-from-Motion, Bundle Adjustment, and available geometry priors to robustly estimate camera pose and automatically reconstruct a fully-dense 3D scene using the least possible number of images in various challenging tunnel-like environments. We validate our system with both Virtual Reality application and experimentation with a real dataset. The results demonstrate that the proposed reconstruction along with texture mapping allows for remote navigation and inspection of tunnel-like environments, even those which are inaccessible for humans.

[1]  Jin Chao,et al.  FaultNet: Faulty Rail-Valves Detection using Deep Learning and Computer Vision , 2019, 2019 IEEE Intelligent Transportation Systems Conference (ITSC).

[2]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[3]  Minh N. Do,et al.  RepMatch: Robust Feature Matching and Pose for Reconstructing Modern Cities , 2016, ECCV.

[4]  Minh N. Do,et al.  Feature-less Stitching of Cylindrical Tunnel , 2018, 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS).

[5]  Simon Fuhrmann,et al.  MVE - A Multi-View Reconstruction Environment , 2014, GCH.

[6]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[7]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[8]  Chee How Tan,et al.  A smart unmanned aerial vehicle (UAV) based imaging system for inspection of deep hazardous tunnels , 2018 .

[9]  Minh N. Do,et al.  Locating 3D Object Proposals: A Depth-Based Online Approach , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Minh N. Do,et al.  Bilateral Functions for Global Motion Modeling , 2014, ECCV.

[11]  Daniel Cremers,et al.  Fast Joint Estimation of Silhouettes and Dense 3D Geometry from Multiple Images , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[13]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[14]  Minh N. Do,et al.  Tracking objects using 3D object proposals , 2017, 2017 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).

[15]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[17]  Shaohui Foong,et al.  Nonlinear Distortion Calibration of an Optical Flow Sensor for Monocular Visual Odometry , 2018, 2018 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM).

[18]  Steven M. Seitz,et al.  Photo Tours , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[19]  Horst Bischof,et al.  Dense reconstruction on-the-fly , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Tarek Hamel,et al.  A UAV for bridge inspection: Visual servoing control law with orientation limits , 2007 .

[21]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[22]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[25]  Julius Ziegler,et al.  StereoScan: Dense 3d reconstruction in real-time , 2011, 2011 IEEE Intelligent Vehicles Symposium (IV).

[26]  David Nister,et al.  Bundle Adjustment Rules , 2006 .

[27]  Robert J. Moorhead,et al.  Using unmanned aerial vehicles for high-resolution remote sensing to map invasive Phragmites australis in coastal wetlands , 2017 .

[28]  Torsten Sattler,et al.  3D Modeling on the Go: Interactive 3D Reconstruction of Large-Scale Scenes on Mobile Devices , 2015, 2015 International Conference on 3D Vision.

[29]  Chee How Tan,et al.  Design Optimization of Sparse Sensing Array for Extended Aerial Robot Navigation in Deep Hazardous Tunnels , 2019, IEEE Robotics and Automation Letters.