Tiny Defect Detection in High-Resolution Aero-Engine Blade Images via a Coarse-to-Fine Framework

This article studies the problem of aero-engine blade surface defect detection in large images. The effective method for aero-engine blade surface inspection for this real application is currently lacking since most defects are relatively small. Therefore, the task of aero-engine blade surface defect detection is mainly implemented by experienced operators, which is subjective and time-consuming. Moreover, it is hard to fit the requirements of higher accuracy and efficiency manually. Therefore, an effective and efficient method for aero-engine blade surface defect detection is demanded. To achieve this, we propose a vision-based framework in this study to detect defects in a coarse-to-fine manner. First, the captured raw images are with a high resolution of $2448\times 2048$ to ensure the accuracy of defect detection. The raw images are then cropped into smaller regions and fed into our deep convolution neural network (DCNN) to learn features with high representation. Next, the coarse classifier module is proposed to filter most background regions out. Finally, the defects are located and classified by a fine detector module in the defective images, which are selected by the coarse classifier module. Instead of directly applying a detector, our coarse-to-fine framework can effectively save computation and improve accuracy. In addition, the coarse-to-fine framework can be trained in an end-to-end manner. Compared with classical methods for object detection, our method also achieves state-of-the-art performance for aero-engine blade surface defect detection in terms of accuracy and efficiency. Furthermore, our framework has been applied for practical application in many aero-engine blade production lines.

[1]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Stephen Pierce,et al.  Reliable thermosonic inspection of aero engine turbine blades , 2010 .

[3]  Andrzej Burghardt,et al.  Robot-Assisted Quality Inspection of Turbojet Engine Blades , 2018 .

[4]  Gyogwon Koo,et al.  Automated defect inspection system for metal surfaces based on deep learning and data augmentation , 2020 .

[5]  Hao Chen,et al.  FCOS: Fully Convolutional One-Stage Object Detection , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Quoc V. Le,et al.  NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jun Wang,et al.  Detecting Trees in Street Images via Deep Learning With Attention Module , 2020, IEEE Transactions on Instrumentation and Measurement.

[8]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[9]  Hua Yang,et al.  An Unsupervised-Learning-Based Approach for Automated Defect Inspection on Textured Surfaces , 2018, IEEE Transactions on Instrumentation and Measurement.

[10]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[11]  Jochen Schlobohm,et al.  Advanced Characterization Techniques for Turbine Blade Wear and Damage , 2017 .

[12]  Li Fei-Fei,et al.  Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Yanbiao Li,et al.  Object Detection and Classification of Metal Polishing Shaft Surface Defects Based on Convolutional Neural Network Deep Learning , 2019, Applied Sciences.

[14]  Mark S. Nixon,et al.  Feature extraction & image processing for computer vision , 2012 .

[15]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[16]  Nima Tajbakhsh,et al.  UNet++: Redesigning Skip Connections to Exploit Multiscale Features in Image Segmentation , 2020, IEEE Transactions on Medical Imaging.

[17]  Jung-Ryul Lee,et al.  Videoscope-based inspection of turbofan engine blades using convolutional neural networks and image processing , 2019 .

[18]  W. K. Wong,et al.  A Statistical Investigation and Optimization of an Industrial Radiography Inspection Process for Aero‐engine Components , 2006, Qual. Reliab. Eng. Int..

[19]  Nuno Vasconcelos,et al.  Cascade R-CNN: Delving Into High Quality Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[20]  Chun-Hung Yeh,et al.  MRtrix3: A fast, flexible and open software framework for medical image processing and visualisation , 2019, NeuroImage.

[21]  Quoc V. Le,et al.  EfficientDet: Scalable and Efficient Object Detection , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[23]  Qinggang Meng,et al.  An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features , 2020, IEEE Transactions on Instrumentation and Measurement.

[24]  Qing He,et al.  Multi-representation adaptation network for cross-domain image classification , 2019, Neural Networks.

[25]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[26]  Yang Lei,et al.  Railway Subgrade Defect Automatic Recognition Method Based on Improved Faster R-CNN , 2018, Sci. Program..

[27]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[28]  Fuqun Zou,et al.  Review of Aero-engine Defect Detection Technology , 2020, 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[29]  Dariusz Szybicki,et al.  ROBOT-OPERATED INSPECTION OF AIRCRAFT ENGINE TURBINE ROTOR GUIDE VANE SEGMENT GEOMETRY , 2017 .

[30]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[31]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Xin Wang,et al.  Few-Shot Object Detection via Feature Reweighting , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[33]  Tao Xiang,et al.  Incremental Few-Shot Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[35]  Di He,et al.  Defect detection of hot rolled steels with a new object detection framework called classification priority network , 2019, Comput. Ind. Eng..

[36]  Junhao Wen,et al.  Fundus Image Classification Using VGG-19 Architecture with PCA and SVD , 2018, Symmetry.

[37]  Shi Yaoyao Measurement techniques of coordinate measuring machine for blade surface of aero-engine , 2012 .

[38]  Anastasios Doulamis,et al.  Pixel-level Corrosion Detection on Metal Constructions by Fusion of Deep Learning Semantic and Contour Segmentation , 2020, ISVC.

[39]  Jure Skvarč,et al.  Segmentation-based deep-learning approach for surface-defect detection , 2019, Journal of Intelligent Manufacturing.

[40]  Longfu Luo,et al.  A Defect-Detection Method of Split Pins in the Catenary Fastening Devices of High-Speed Railway Based on Deep Learning , 2020, IEEE Transactions on Instrumentation and Measurement.

[41]  Ben Poole,et al.  Categorical Reparameterization with Gumbel-Softmax , 2016, ICLR.

[42]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[43]  Integrative Solution for In-situ Ultrasonic Inspection of Aero-engine Blades Using Endoscopic Cheap Optical Transducers ( CHOTs ) , .

[44]  Liqiong Chen,et al.  Feature weighting network for aircraft engine defect detection , 2020, Int. J. Wavelets Multiresolution Inf. Process..

[45]  T. Jayakumar,et al.  Dual-frequency Eddy Current Non-destructive Detection of Fatigue Cracks in Compressor Discs of Aero Engines , 2004 .

[46]  Kan Chen,et al.  Billion-scale semi-supervised learning for image classification , 2019, ArXiv.

[47]  Qian Xie,et al.  Automatic defect detection of metro tunnel surfaces using a vision-based inspection system , 2021, Adv. Eng. Informatics.

[48]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[49]  Guoan Yang,et al.  Prediction of Fatigue Crack Growth in Gas Turbine Engine Blades Using Acoustic Emission , 2018, Sensors.

[50]  Dawei Li,et al.  Automatic Detection and Classification of Sewer Defects via Hierarchical Deep Learning , 2019, IEEE Transactions on Automation Science and Engineering.

[51]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[52]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[54]  Yu-Wing Tai,et al.  Few-Shot Object Detection With Attention-RPN and Multi-Relation Detector , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Ujjwal Maulik,et al.  Understanding Deep Learning Techniques for Image Segmentation , 2019, ACM Comput. Surv..

[56]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Zhi Zhang,et al.  Bag of Tricks for Image Classification with Convolutional Neural Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).