Integrated design of an aerial soft-continuum manipulator for predictive maintenance

This article presents an integrated concept of an aerial robot used for predictive maintenance in the construction sector. The latter can be remotely controlled, allowing the localization of cracks on wall surfaces and the adaptive deposit of the material for in situ repairs. The use of an aerial robot is motivated by fast intervention, allowing time and cost minimizing of overhead repairs without the need for scaffolding. It is composed of a flying mobile platform positioned in stationary mode to guide a soft continuum arm that allows to reach the area of cracks with different access points. Indeed, some constructions have complex geometries that present problems for access using rigid mechanical arms. The aerial robot uses visual sensors to automatically identify and localize cracks in walls, based on deep learning convolutional neural networks. A centerline representing the structural feature of the crack is computed. The soft continuum manipulator is used to guide the continuous deposit of the putty material to fill the microscopic crack. For this purpose, an inverse kinematic model-based control of the soft arm is developed, allowing to estimate the length of the bending tubes. The latter are then used as inputs for a neural network to predict the desired input pressure to bend the actuated soft tubes. A set of experiments was carried out on cracks located on flat and oblique surfaces, to evaluate the actual performances of the predictive maintenance mechatronic robot.

[1]  Alexander Rassau,et al.  Score-based mask edge improvement of Mask-RCNN for segmentation of fruit and vegetables , 2021, Expert Syst. Appl..

[2]  M. Nehdi,et al.  Classification and quantification of cracks in concrete structures using deep learning image-based techniques , 2020 .

[3]  Seung-Hyun Eem,et al.  Concrete crack detection and quantification using deep learning and structured light , 2020 .

[4]  Yuyu Tian,et al.  Detection and segmentation of overlapped fruits based on optimized mask R-CNN application in apple harvesting robot , 2020, Comput. Electron. Agric..

[5]  Kaspar Althoefer,et al.  Implementing Tactile and Proximity Sensing for Crack Detection , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Amir Mosavi,et al.  Deep Learning for Detecting Building Defects Using Convolutional Neural Networks , 2019, Sensors.

[7]  Zhun Fan,et al.  Design and Implementation of a Manipulator System for Roadway Crack Sealing , 2019, 2019 IEEE 9th Annual International Conference on CYBER Technology in Automation, Control, and Intelligent Systems (CYBER).

[8]  Yang Yang,et al.  Deep-Learning-Based Bughole Detection for Concrete Surface Image , 2019, Advances in Civil Engineering.

[9]  Wei Wu,et al.  Multi-Object Tracking with Multiple Cues and Switcher-Aware Classification , 2019, ArXiv.

[10]  Guan Huang,et al.  Attention-Guided Unified Network for Panoptic Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yugyung Lee,et al.  Utilizing Mask R-CNN for Detection and Segmentation of Oral Diseases , 2018, 2018 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[12]  Robert J. Thomas,et al.  SDNET2018: An annotated image dataset for non-contact concrete crack detection using deep convolutional neural networks , 2018, Data in brief.

[13]  Robert J. Thomas,et al.  Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete , 2018, Construction and Building Materials.

[14]  Rochdi Merzouki,et al.  Adaptive Algorithms for Performance Improvement of a Class of Continuum Manipulators , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[15]  Ã.F. Ãzgenel,et al.  Performance Comparison of Pretrained Convolutional Neural Networks on Crack Detection in Buildings , 2018 .

[16]  Fatih Kucuksubasi,et al.  Transfer Learning-Based Crack Detection by Autonomous UAVs , 2018, Proceedings of the 35th International Symposium on Automation and Robotics in Construction (ISARC).

[17]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[18]  Wilson Ricardo Leal da Silva,et al.  Concrete Cracks Detection Based on Deep Learning Image Classification , 2018, Proceedings.

[19]  Robert J. Thomas,et al.  Deep Learning Neural Networks for sUAS-Assisted Structural Inspections: Feasibility and Application , 2018, 2018 International Conference on Unmanned Aircraft Systems (ICUAS).

[20]  Marc Maguire,et al.  Bridge inspection: human performance, unmanned aerial systems and automation , 2018, Journal of Civil Structural Health Monitoring.

[21]  I-Ming Chen,et al.  A novel building post-construction quality assessment robot: Design and prototyping , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[22]  Tran Hiep Dinh,et al.  Automatic Crack Detection in Built Infrastructure Using Unmanned Aerial Vehicles , 2017, ArXiv.

[23]  Calvin Coopmans,et al.  Challenges in bridge inspection using small unmanned aerial systems: Results and lessons learned , 2017, 2017 International Conference on Unmanned Aircraft Systems (ICUAS).

[24]  Hao Jiang,et al.  A two-level approach for solving the inverse kinematics of an extensible soft arm considering viscoelastic behavior , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[25]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[26]  Sumathi Poobal,et al.  Crack detection using image processing: A critical review and analysis , 2017, Alexandria Engineering Journal.

[27]  Jochen J. Steil,et al.  Hybrid Analytical and Data-Driven Modeling for Feed-Forward Robot Control † , 2017, Sensors.

[28]  Fan Meng,et al.  Automatic Road Crack Detection Using Random Structured Forests , 2016, IEEE Transactions on Intelligent Transportation Systems.

[29]  Kin Huat Low,et al.  Innovations in Infrastructure Service Robots , 2016 .

[30]  Rochdi Merzouki,et al.  Hybrid Approach for Modeling and Solving of Kinematics of a Compact Bionic Handling Assistant Manipulator , 2016, IEEE/ASME Transactions on Mechatronics.

[31]  Kristin J. Dana,et al.  Automated Crack Detection on Concrete Bridges , 2016, IEEE Transactions on Automation Science and Engineering.

[32]  Pushparaj Mani Pathak,et al.  Kinematic Calibration of a Multisection Bionic Manipulator , 2015, IEEE/ASME Transactions on Mechatronics.

[33]  Rochdi Merzouki,et al.  Adaptive Neural Network Control of a Compact Bionic Handling Arm , 2015, IEEE/ASME Transactions on Mechatronics.

[34]  Yun Liu,et al.  Automatic Crack Detection and Classification Method for Subway Tunnel Safety Monitoring , 2014, Sensors.

[35]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[36]  Anthony Yezzi,et al.  Automating the Crack Map Detection Process for Machine Operated Crack Sealer , 2013 .

[37]  Eduardo Castillo Castañeda,et al.  Crack detection by a climbing robot using image analysis , 2013, CONIELECOMP 2013, 23rd International Conference on Electronics, Communications and Computing.

[38]  Paulo Lobato Correia,et al.  Automatic Road Crack Detection and Characterization , 2013, IEEE Transactions on Intelligent Transportation Systems.

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  O. M. Ramahi,et al.  Complementary Split-Ring Resonator for Crack Detection in Metallic Surfaces , 2012, IEEE Microwave and Wireless Components Letters.

[41]  Boubaker Daachi,et al.  Adaptive neural controller for redundant robot manipulators and collision avoidance with mobile obstacles , 2012, Neurocomputing.

[42]  Theodore E. Matikas,et al.  NDT approach for characterization of subsurface cracks in concrete , 2011 .

[43]  Kin-tak Lau,et al.  Fibre-optic sensors and smart composites for concrete applications , 2003 .

[44]  Serkan Aydin,et al.  An improved approach to the solution of inverse kinematics problems for robot manipulators , 2000 .

[45]  Robert J. Thomas,et al.  SDNET2018: A concrete crack image dataset for machine learning applications , 2018 .

[46]  Rochdi Merzouki,et al.  Qualitative approach for forward kinematic modeling of a Compact Bionic Handling Assistant trunk , 2014 .