Contaminated Facade Identification Using Convolutional Neural Network and Image Processing

In recent years, as number of new building getting larger, there has been an increased interest in the cleaning of exterior walls. Accordingly, there is a growing interest in automatic cleaning robots that move around the outer building façade. These robots are also required to apply different cleaning methods to remove various contaminants on the outer wall of the building. However, current surface contaminant detection systems can either detect only a single type of contaminant, or are not compact enough for installation on mobile platforms that move around the outer façade. As cleaning workers are able to distinguish various contaminants with the naked eye, we aim to solve this problem by developing a machine-vision system using convolutional neural networks (CNNs) and image processing methods. As it is a compact system that uses only a camera to take pictures and a processor to process the images, it is suitable for applications involving mobile platforms. Object-type contaminants such as avian feces are handled by the YOLOv3 module using the object-detection algorithm. Area-type contaminants such as rusty stains are processed using the color-detection module using the HSV color space, median filter, and flood fill algorithm. Particle-type contaminants such as dust are handled by the grayscale module, converting images to grayscale images and then comparing the average brightness with a reference that is provided in advance. This proposed machine vision system will detect objects, areas, and particle-type contaminants with a single image and some reference images provided in advance.

[1]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Chenglin Wang,et al.  Real-time detection of surface deformation and strain in recycled aggregate concrete-filled steel tubular columns via four-ocular vision , 2019, Robotics Comput. Integr. Manuf..

[3]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[4]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Christine Connolly,et al.  A study of efficiency and accuracy in the transformation from RGB to CIELAB color space , 1997, IEEE Trans. Image Process..

[6]  Bo Du,et al.  Dimensionality Reduction With Enhanced Hybrid-Graph Discriminant Learning for Hyperspectral Image Classification , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Performance Analysis of Various Filters for Image Noise Removal in Different Noise Environment , 2014 .

[8]  Aleksandra Kawala-Janik,et al.  YUV vs RGB-Choosing a Color Space for Human-Machine Interaction , 2014, FedCSIS.

[9]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jiamin Liu,et al.  Local Geometric Structure Feature for Dimensionality Reduction of Hyperspectral Imagery , 2017, Remote. Sens..

[11]  Luc Van Gool,et al.  Efficient Non-Maximum Suppression , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[12]  Bart De Schutter,et al.  Deep convolutional neural networks for detection of rail surface defects , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[13]  Sungon Lee,et al.  Robust Design of Detecting Contaminants in Façade Cleaning Applications , 2020, IEEE Access.

[14]  Marcus Strand,et al.  Classification of 3D structures based on an object detection for facade elements in multiple views during the reconstruction process , 2019, 2019 IEEE International Conference on Industrial Cyber Physical Systems (ICPS).

[15]  Houxiang Zhang,et al.  Robotic cleaning system for glass facade of high-rise airport control tower , 2010, Ind. Robot.

[16]  E. Gambao,et al.  Control System for a Semi-automatic Façade Cleaning Robot , 2006 .

[17]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[18]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[19]  Jianwei Zhang,et al.  A series of pneumatic glass-wall cleaning robots for high-rise buildings , 2007, Ind. Robot.

[20]  Dale A. Richter,et al.  Short-range noncontact detection of surface contamination using Raman lidar , 2002, SPIE Optics East.

[21]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[22]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[23]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Vishal Mandal,et al.  Automated Road Crack Detection Using Deep Convolutional Neural Networks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[25]  Jianke Zhu,et al.  DeepFacade: A Deep Learning Approach to Facade Parsing With Symmetric Loss , 2020, IEEE Transactions on Multimedia.

[26]  Daehie Hong,et al.  Building wall maintenance robot based on built-in guide rail , 2012, 2012 IEEE International Conference on Industrial Technology.

[27]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Imtiaz Ahmed Choudhury,et al.  Application of Taguchi method in the optimization of end milling parameters , 2004 .

[29]  Norbert Elkmann,et al.  Kinematics, sensors and control of the fully automated façade-cleaning robot SIRIUSc for the Fraunhofer headquarters building, Munich , 2008, Ind. Robot.

[30]  Chris Dyer,et al.  Short-range remote detection of liquid surface contamination by active imaging Fourier transform spectrometry. , 2008, Optics express.

[31]  JongWon Kim,et al.  Survey on Glass And Façade-Cleaning Robots: Climbing Mechanisms, Cleaning Methods, and Applications , 2019, International Journal of Precision Engineering and Manufacturing-Green Technology.

[32]  Rajesh Kumar,et al.  Application of Taguchi Method for Optimizing Turning Process by the effects of Machining Parameters , 2012 .

[33]  Jarmo T. Alander,et al.  Integer-based accurate conversion between RGB and HSV color spaces , 2015, Comput. Electr. Eng..

[34]  Kuang-Hui Chi,et al.  A Deep Learning Application for Detecting Facade Tile Degradation , 2019, IHSED.

[35]  Anil Kumar Gupta,et al.  Comparing the Performance of L*A*B* and HSV Color Spaces with Respect to Color Image Segmentation , 2015, ArXiv.

[36]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.