Automatic visual defects inspection of wind turbine blades via YOLO-based small object detection approach

Abstract. Regular inspection of wind turbine blades (WTBs), especially the detection of tiny defects, is necessary to maintain safe operation of wind turbine systems. However, current detections are inefficient and subjective because they are conducted merely by human inspectors. An autonomous visual inspection system is proposed in this paper for WTBs, in which a deep learning framework is developed by combining the convolutional neural network (CNN) and the you only look once (YOLO) model. To achieve practically acceptable detection accuracy for small-sized defects on the WTBs, a YOLO-based small object detection approach (YSODA) using a multiscale feature pyramid is proposed by amalgamating features of more layers. To evaluate the proposed YSODA, a database including 23,807 images labeled for three types of defect—crack, oil pollution, and sand inclusion, is developed. Then, the YSODA is with its architecture modified, and is trained, validated, and tested using the images from the database to provide autonomous and accurate visual inspection. After training and testing, resulting detection accuracy reaches 92.7%, 90.7%, and 90.3% for the three types of defect with the average accuracy being 91.3%. The robustness of the trained YSODA is demonstrated and verified in detecting small-sized defects. It is also compared with that of the traditional CNN-based and machine learning methods by applying to a real WTB system, which proved that the proposed YSODA is superior to existing approaches in terms of detection accuracy and reliability.

[1]  Amy Loutfi,et al.  Classification and Segmentation of Satellite Orthoimagery Using Convolutional Neural Networks , 2016, Remote. Sens..

[2]  Zijun Zhang,et al.  Automatic Detection of Wind Turbine Blade Surface Cracks Based on UAV-Taken Images , 2017, IEEE Transactions on Industrial Electronics.

[3]  Ronald M. Summers,et al.  Improving Computer-Aided Detection Using Convolutional Neural Networks and Random View Aggregation , 2015, IEEE Transactions on Medical Imaging.

[4]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[5]  Hayit Greenspan,et al.  Multi-view longitudinal CNN for multiple sclerosis lesion segmentation , 2017, Eng. Appl. Artif. Intell..

[6]  A. G. Dutton,et al.  Acoustic Emission Monitoring of Small Wind Turbine Blades , 2002 .

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Gangbing Song,et al.  Wind turbine blade health monitoring with piezoceramic-based wireless sensor network , 2013 .

[9]  Jae-Kyung Lee,et al.  Transformation algorithm of wind turbine blade moment signals for blade condition monitoring , 2015 .

[10]  Shuo Yang,et al.  Image-based damage recognition of wind turbine blades , 2017, 2017 2nd International Conference on Advanced Robotics and Mechatronics (ICARM).

[11]  Antidio Viguria,et al.  Thermographic non-destructive inspection of wind turbine blades using unmanned aerial systems , 2015 .

[12]  Renaldas Raisutis,et al.  Hybrid Signal Processing Technique to Improve the Defect Estimation in Ultrasonic Non-Destructive Testing of Composite Structures , 2017, Sensors.

[13]  Jianping Chen,et al.  Study on visual detection method for wind turbine blade failure , 2018 .

[14]  Young-Chon Kim,et al.  In-Situ Cure Monitoring of Wind Turbine Blades by Using Fiber Bragg Grating Sensors and Fresnel Reflection Measurement , 2015, Sensors.

[15]  John K. Jackman,et al.  Feasibility of Automatic Detection of Surface Cracks in Wind Turbine Blades , 2014 .

[16]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[17]  Young-Shin Lee,et al.  Damage assessment of wind turbine blade under static loading test using acoustic emission , 2014 .

[18]  Pak Sing Leung,et al.  Wavelet transform-based stress-time history editing of horizontal axis wind turbine blades , 2014 .

[19]  Jun Liu,et al.  Detection and analysis of large-scale WT blade surface cracks based on UAV-taken images , 2018, IET Image Process..