An integrated UGV-UAV system for construction site data collection

Abstract There have been recent efforts to increase the degree of automation and frequency of data collection for construction applications using Unmanned Aerial/Ground Vehicles (UAV/UGV). However, the current practice of data collection is traditionally performed, which is manual, costly, time-consuming, and error-prone. Developing vision-based mobile robotic systems that are aware of its surrounding and capable of autonomous navigation are becoming essential to many construction applications, namely surveying, monitoring, and inspection. Nevertheless, the systems above suffer from a series of performance issues. One major problem is inefficient navigation in indoor and cluttered scenes with many obstacles and barriers, where some places are inaccessible by a UGV. To provide a solution to this problem, this paper designs a UAV-UGV team that integrates two custom-built mobile robots. The UGV autonomously navigates through space, leveraging its sensors. The UAV acts as an external eye for the UGV, observing the scene from a vantage point that is inaccessible to the UGV. The relative pose of the UAV is estimated continuously, which allows it to maintain a fixed location that is relative to the UGV. The key aspects for the development of this system that is capable of autonomous navigation are the localization of both UAV and UGV, mapping of the surrounding environment, and efficient path planning using multiple sensors. The proposed system is tested in an indoor and cluttered construction-like environment. The performance of the system demonstrates the feasibility of developing and deploying a robust and automated data collection system for construction applications in the near future.

[1]  Chang-Soo Han,et al.  Feasibility verification of brick-laying robot using manipulation trajectory and the laying pattern optimization , 2009 .

[2]  Anthony G. Cohn,et al.  Localisation of a mobile robot for bridge bearing inspection , 2018, Automation in Construction.

[3]  Hubert Roth,et al.  Ground Robot Navigation Relies on Blimp Vision Information , 2014 .

[4]  Thomas Bock,et al.  The future of construction automation: Technological disruption and the upcoming ubiquity of robotics , 2015 .

[5]  Alessandro De Gloria,et al.  Air-ground multi-agent robot team coordination , 2011, 2011 IEEE International Conference on Robotics and Automation.

[6]  Ming Gu,et al.  The IFC-based path planning for 3D indoor spaces , 2013, Adv. Eng. Informatics.

[7]  Wei Meng,et al.  Autonomous Exploration and Mapping System Using Heterogeneous UAVs and UGVs in GPS-Denied Environments , 2019, IEEE Transactions on Vehicular Technology.

[8]  Vijay Kumar,et al.  Synergies in Feature Localization by Air-Ground Robot Teams , 2004, ISER.

[9]  Antonio Barrientos,et al.  An Aerial–Ground Robotic System for Navigation and Obstacle Mapping in Large Outdoor Areas , 2013, Sensors.

[10]  Mojtaba Noghabaei,et al.  Real-Time Image Localization and Registration with BIM Using Perspective Alignment for Indoor Monitoring of Construction , 2019, J. Comput. Civ. Eng..

[11]  Rahul Jain,et al.  Vision-based Obstacle Removal System for Autonomous Ground Vehicles Using a Robotic Arm , 2019, Computing in Civil Engineering 2019.

[12]  Javier Civera,et al.  S-PTAM: Stereo Parallel Tracking and Mapping , 2017, Robotics Auton. Syst..

[13]  François Michaud,et al.  RTAB‐Map as an open‐source lidar and visual simultaneous localization and mapping library for large‐scale and long‐term online operation , 2018, J. Field Robotics.

[14]  OesterreichThuy Duong,et al.  Understanding the implications of digitisation and automation in the context of Industry 4.0 , 2016 .

[15]  Edgar J. Lobaton,et al.  Building an Integrated Mobile Robotic System for Real-Time Applications in Construction , 2018, Proceedings of the 35th International Symposium on Automation and Robotics in Construction (ISARC).

[16]  Mani Golparvar-Fard,et al.  Visual monitoring of civil infrastructure systems via camera-equipped Unmanned Aerial Vehicles (UAVs): a review of related works , 2016 .

[17]  J. M. M. Montiel,et al.  ORB-SLAM: A Versatile and Accurate Monocular SLAM System , 2015, IEEE Transactions on Robotics.

[18]  Thejus Pathmakumar,et al.  Floor cleaning robot with reconfigurable mechanism , 2018, Automation in Construction.

[19]  Stjepan Bogdan,et al.  Decentralized planning and control for UAV–UGV cooperative teams , 2018, Autonomous Robots.

[20]  Shaojie Shen,et al.  VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator , 2017, IEEE Transactions on Robotics.

[21]  Alessandro De Gloria,et al.  Multi-robot search and rescue team , 2011, 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics.

[22]  Tao Xu,et al.  Onboard controlling system design of unmanned airship , 2011, Proceedings of 2011 International Conference on Electronic & Mechanical Engineering and Information Technology.

[23]  Clément Gosselin,et al.  Large-scale 3D printing of ultra-high performance concrete – a new processing route for architects and builders , 2016 .

[24]  Yong K. Cho,et al.  A BIM and UWB integrated Mobile Robot Navigation System for Indoor Position Tracking Applications , 2016 .

[25]  Kazuya Yoshida,et al.  Collaborative mapping of an earthquake‐damaged building via ground and aerial robots , 2012, J. Field Robotics.

[26]  Suresh Sundaram,et al.  Autonomous Navigation and Sensorless Obstacle Avoidance for UGV with Environment Information from UAV , 2018, 2018 Second IEEE International Conference on Robotic Computing (IRC).

[27]  Dorian Gálvez-López,et al.  Bags of Binary Words for Fast Place Recognition in Image Sequences , 2012, IEEE Transactions on Robotics.

[28]  Benjamin Grocholsky,et al.  Integrated long-range UAV/UGV collaborative target tracking , 2009, Defense + Commercial Sensing.

[29]  Davide Scaramuzza,et al.  SVO: Fast semi-direct monocular visual odometry , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Dezhen Song,et al.  Kinematic Modeling and Analysis of Skid-Steered Mobile Robots With Applications to Low-Cost Inertial-Measurement-Unit-Based Motion Estimation , 2009, IEEE Transactions on Robotics.

[31]  Anjan Chakrabarty,et al.  Autonomous indoor object tracking with the Parrot AR.Drone , 2016, 2016 International Conference on Unmanned Aircraft Systems (ICUAS).

[32]  Martin Becker,et al.  MULTISENSOR EQUIPPED UAV/UGV FOR AUTOMATED EXPLORATION , 2017 .

[33]  Youmin Zhang,et al.  Real-time autonomous take-off, tracking and landing of UAV on a moving UGV platform , 2016, 2016 24th Mediterranean Conference on Control and Automation (MED).

[34]  Jean-François Brethé,et al.  A decentralized interactive architecture for aerial and ground mobile robots cooperation , 2015, 2015 International Conference on Control, Automation and Robotics.

[35]  Roland Siegwart,et al.  Collaborative navigation for flying and walking robots , 2016, IROS 2016.

[36]  Luiz Chaimowicz,et al.  Deploying Air-Ground Multi-Robot Teams in Urban Environments , 2005 .

[37]  Cheng Hui,et al.  Autonomous takeoff, tracking and landing of a UAV on a moving UGV using onboard monocular vision , 2013, Proceedings of the 32nd Chinese Control Conference.

[38]  Frank Teuteberg,et al.  Understanding the implications of digitisation and automation in the context of Industry 4.0: A triangulation approach and elements of a research agenda for the construction industry , 2016, Comput. Ind..

[39]  Andrew B. Kahng,et al.  Cooperative Mobile Robotics: Antecedents and Directions , 1997, Auton. Robots.

[40]  Caitlin Mueller,et al.  Feasibility study for drone-based masonry construction of real-scale structures , 2018, Automation in Construction.

[41]  Libor Preucil,et al.  A Practical Multirobot Localization System , 2014, J. Intell. Robotic Syst..

[42]  Raj Madhavan,et al.  Registration of range data from unmanned aerial and ground vehicles , 2003, 32nd Applied Imagery Pattern Recognition Workshop, 2003. Proceedings..

[43]  Dirk Merkel,et al.  Docker: lightweight Linux containers for consistent development and deployment , 2014 .

[44]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[45]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[46]  S. LaValle Rapidly-exploring random trees : a new tool for path planning , 1998 .

[47]  Wolfram Burgard,et al.  OctoMap: an efficient probabilistic 3D mapping framework based on octrees , 2013, Autonomous Robots.

[48]  Andrzej Więckowski,et al.  “JA-WA” - A wall construction system using unilateral material application with a mobile robot , 2017 .

[49]  Roland Siegwart,et al.  Vision based MAV navigation in unknown and unstructured environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[50]  Ioannis Brilakis,et al.  Automated re-prefabrication system for buildings using robotics , 2017 .

[51]  Vijay Kumar,et al.  Cooperative air and ground surveillance , 2006, IEEE Robotics & Automation Magazine.

[52]  Jiwon Seo,et al.  Multi-UAV-based stereo vision system without GPS for ground obstacle mapping to assist path planning of UGV , 2014 .

[53]  Edgar Lobaton,et al.  LNSNet: Lightweight Navigable Space Segmentation for Autonomous Robots on Construction Sites , 2019, Data.

[54]  Anastasios I. Mourikis,et al.  High-precision, consistent EKF-based visual-inertial odometry , 2013, Int. J. Robotics Res..

[55]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[56]  Gordon Christie,et al.  Radiation search operations using scene understanding with autonomous UAV and UGV , 2016, J. Field Robotics.

[57]  Jason M. O'Kane,et al.  Bird's eye view: Cooperative exploration by UGV and UAV , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[58]  Flavio Fontana,et al.  Aerial-guided navigation of a ground robot among movable obstacles , 2014, 2014 IEEE International Symposium on Safety, Security, and Rescue Robotics (2014).

[59]  João Valente,et al.  Towards a ground navigation system based in visual feedback provided by a mini UAV , 2012 .

[60]  Josechu J. Guerrero,et al.  Dense RGB-D visual odometry using inverse depth , 2016, Robotics Auton. Syst..

[61]  Javier V. Gómez,et al.  Pilot-Scale Development of a UAV-UGV Hybrid with Air-Based UGV Path Planning , 2012, 2012 10th International Conference on Frontiers of Information Technology.

[62]  Low Sui Pheng,et al.  Managing productivity in construction: JIT operations and measurements , 1997 .

[63]  Roland Siegwart,et al.  Maplab: An Open Framework for Research in Visual-Inertial Mapping and Localization , 2017, IEEE Robotics and Automation Letters.

[64]  Meng-Han Tsai,et al.  A review of rotorcraft Unmanned Aerial Vehicle (UAV) developments and applications in civil engineering , 2014 .

[65]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[66]  Wolfram Burgard,et al.  3-D Mapping With an RGB-D Camera , 2014, IEEE Transactions on Robotics.

[67]  Matthias Nießner,et al.  State of the Art on 3D Reconstruction with RGB‐D Cameras , 2018, Comput. Graph. Forum.

[68]  Eugenio Culurciello,et al.  ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation , 2016, ArXiv.

[69]  Yong K. Cho,et al.  Case Study of BIM and Cloud–Enabled Real-Time RFID Indoor Localization for Construction Management Applications , 2016 .

[70]  Davide Scaramuzza,et al.  Air-ground localization and map augmentation using monocular dense reconstruction , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[71]  Edgar Lobaton,et al.  Vision-based integrated mobile robotic system for real-time applications in construction , 2018, Automation in Construction.

[72]  F. Michaud,et al.  Appearance-Based Loop Closure Detection for Online Large-Scale and Long-Term Operation , 2013, IEEE Transactions on Robotics.

[73]  John Peterson,et al.  Online Aerial Terrain Mapping for Ground Robot Navigation , 2018, Sensors.

[74]  Juho Kannala,et al.  DT-SLAM: Deferred Triangulation for Robust SLAM , 2014, 2014 2nd International Conference on 3D Vision.

[75]  Emilio Frazzoli,et al.  Sampling-based algorithms for optimal motion planning , 2011, Int. J. Robotics Res..

[76]  Fabio Gramazio,et al.  Building a Bridge with Flying Robots , 2016 .

[77]  T. Krajnik,et al.  Cooperative μUAV-UGV autonomous indoor surveillance , 2012, International Multi-Conference on Systems, Sygnals & Devices.

[78]  Youngjib Ham,et al.  Automated content-based filtering for enhanced vision-based documentation in construction toward exploiting big visual data from drones , 2019, Automation in Construction.

[79]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[80]  Daniel Cremers,et al.  Dense visual SLAM for RGB-D cameras , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[81]  Huili Yu,et al.  Moving ground target tracking in urban terrain using air/ground vehicles , 2010, 2010 IEEE Globecom Workshops.

[82]  Carlos Balaguer,et al.  Tunnel structural inspection and assessment using an autonomous robotic system , 2018 .