Deep learning for industrial image: challenges, methods for enriching the sample space and restricting the hypothesis space, and possible issue

Deep learning (DL) is an important enabling technology for intelligent manufacturing. The DL-based industrial image pattern recognition (DLBIIPR) plays a vital role in the improvement of product qu...

[1]  Xu Ke,et al.  Surface defect classification of steels with a new semi-supervised learning method , 2019, Optics and Lasers in Engineering.

[2]  Roberto Cipolla,et al.  Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Qingyong Li,et al.  A Hierarchical Extractor-Based Visual Rail Surface Inspection System , 2017, IEEE Sensors Journal.

[4]  Sheng-Fuu Lin,et al.  X-ray inspection image enhencement using shearlet transform , 2012, 2012 7th IEEE Conference on Industrial Electronics and Applications (ICIEA).

[5]  Bohyung Han,et al.  Learning Multi-domain Convolutional Neural Networks for Visual Tracking , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Yuji Iwahori,et al.  Defect Classification of Electronic Board Using Dense SIFT and CNN , 2018, KES.

[7]  Wuyi Ming,et al.  Defect detection of LGP based on combined classifier with dynamic weights , 2019, Measurement.

[8]  Yunze He,et al.  Joint Scanning Laser Thermography Defect Detection Method for Carbon Fiber Reinforced Polymer , 2020, IEEE Sensors Journal.

[9]  Yujie Li,et al.  Railway track fastener defect detection based on image processing and deep learning techniques: A comparative study , 2019, Eng. Appl. Artif. Intell..

[10]  Thomas Brox,et al.  Striving for Simplicity: The All Convolutional Net , 2014, ICLR.

[11]  Laith Mohammad Abualigah,et al.  (AIAM2019) Artificial Intelligence in Software Engineering and inverse: Review , 2020, Int. J. Comput. Integr. Manuf..

[12]  Xiao Chen,et al.  Automatic detection of photovoltaic module defects in infrared images with isolated and develop-model transfer deep learning , 2020 .

[13]  Chen Wang,et al.  SSIM-NET: Real-Time PCB Defect Detection Based on SSIM and MobileNet-V3 , 2019, 2019 2nd World Conference on Mechanical Engineering and Intelligent Manufacturing (WCMEIM).

[14]  Jinxiu Ma,et al.  Fabric Defect Detection Using L0 Gradient Minimization and Fuzzy C-Means , 2019, Applied Sciences.

[15]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Kai Chen,et al.  A CNN Model Based on Spatial Attention Modules for Casting Type Classification on Pseudo-color Digital Radiography Images , 2019, 2019 Chinese Automation Congress (CAC).

[17]  Xiuqiao Xiang,et al.  Aircraft detection in remote sensing image based on corner clustering and deep learning , 2020, Eng. Appl. Artif. Intell..

[18]  Hui Lin,et al.  Defect Image Sample Generation With GAN for Improving Defect Recognition , 2020, IEEE Transactions on Automation Science and Engineering.

[19]  Xiaoou Li,et al.  Deformable Convolutional Networks for Efficient Mixed-Type Wafer Defect Pattern Recognition , 2020, IEEE Transactions on Semiconductor Manufacturing.

[20]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Guangrui Wen,et al.  Weld image deep learning-based on-line defects detection using convolutional neural networks for Al alloy in robotic arc welding , 2019, Journal of Manufacturing Processes.

[23]  Shuicheng Yan,et al.  Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks With Octave Convolution , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[24]  Shuzhi Sam Ge,et al.  Defect identification of wind turbine blades based on defect semantic features with transfer feature extractor , 2020, Neurocomputing.

[25]  Yu He,et al.  Unified detection method of aluminium profile surface defects: Common and rare defect categories , 2020 .

[26]  Junliang Wang,et al.  AdaBalGAN: An Improved Generative Adversarial Network With Imbalanced Learning for Wafer Defective Pattern Recognition , 2019, IEEE Transactions on Semiconductor Manufacturing.

[27]  Guanghua Xu,et al.  Health indicator construction of machinery based on end-to-end trainable convolution recurrent neural networks , 2020 .

[28]  Shibin Gao,et al.  Deep Architecture for High-Speed Railway Insulator Surface Defect Detection: Denoising Autoencoder With Multitask Learning , 2019, IEEE Transactions on Instrumentation and Measurement.

[29]  A. Mesbah,et al.  Deep Learning for Classification of the Chemical Composition of Particle Defects on Semiconductor Wafers , 2020, IEEE Transactions on Semiconductor Manufacturing.

[30]  Ying Liu,et al.  Application of deep convolutional neural network on feature extraction and detection of wood defects , 2020 .

[31]  Jure Skvarč,et al.  Segmentation-based deep-learning approach for surface-defect detection , 2019, Journal of Intelligent Manufacturing.

[32]  Madan M. Gupta,et al.  On Definition of Deep Learning , 2018, 2018 World Automation Congress (WAC).

[33]  Guojun Wen,et al.  Automatic Fabric Defect Detection with a Multi-Scale Convolutional Denoising Autoencoder Network Model , 2018, Sensors.

[34]  Gopalakrishnan Sethumadhavan,et al.  Anisotropic diffusion based denoising on X-radiography images to detect weld defects , 2017, Digit. Signal Process..

[35]  Lining Sun,et al.  Defect Enhancement Generative Adversarial Network for Enlarging Data Set of Microcrack Defect , 2019, IEEE Access.

[36]  Dimitrios Kosmopoulos,et al.  Multiclass defect detection and classification in weld radiographic images using geometric and texture features , 2010, Expert Syst. Appl..

[37]  Sheng Liu,et al.  Defect detection of nuclear fuel assembly based on deep neural network , 2020 .

[38]  Le Wang,et al.  A Rail Fastener Tightness Detection Approach Using Multi-source Visual Sensor , 2020, Sensors.

[39]  Yu He,et al.  Semi-supervised defect classification of steel surface based on multi-training and generative adversarial network , 2019, Optics and Lasers in Engineering.

[40]  Jinwu Xu,et al.  Defects identification using the improved ultrasonic measurement model and support vector machines , 2020 .

[41]  Franco Turini,et al.  A Survey of Methods for Explaining Black Box Models , 2018, ACM Comput. Surv..

[42]  Qingquan Song,et al.  Towards Explanation of DNN-based Prediction with Guided Feature Inversion , 2018, KDD.

[43]  Yongxiong Wang,et al.  An Efficient Convolutional Neural Network Model Based on Object-Level Attention Mechanism for Casting Defect Detection on Radiography Images , 2020, IEEE Transactions on Industrial Electronics.

[44]  De Xu,et al.  Automatic Metallic Surface Defect Detection and Recognition with Convolutional Neural Networks , 2018, Applied Sciences.

[45]  Jack Beuth,et al.  A multi-scale convolutional neural network for autonomous anomaly detection and classification in a laser powder bed fusion additive manufacturing process , 2018, Additive Manufacturing.

[46]  Yaonan Wang,et al.  In-line inspection solution for codes on complex backgrounds for the plastic container industry , 2019 .

[47]  Xiangyu Zhang,et al.  ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[48]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Nan Chen,et al.  Wafer Map Defect Pattern Recognition Using Rotation-Invariant Features , 2019, IEEE Transactions on Semiconductor Manufacturing.

[50]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  Ming-Hsuan Yang,et al.  Hierarchical Convolutional Features for Visual Tracking , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[52]  Di He,et al.  Design of multi-scale receptive field convolutional neural network for surface inspection of hot rolled steels , 2019, Image Vis. Comput..

[53]  Ge Zhang,et al.  Approaches for improvement of the X-ray image defect detection of automobile casting aluminum parts based on deep learning , 2019, NDT & E International.

[54]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Xiaokai Wang,et al.  Quantitative ultrasonic testing for near-surface defects of large ring forgings using feature extraction and GA-SVM , 2021 .

[56]  Jixiu Wu,et al.  Automatic detonator code recognition via deep neural network , 2020, Expert Syst. Appl..

[57]  Michael Ying Yang,et al.  A deep-learning-based approach for fast and robust steel surface defects classification , 2019, Optics and Lasers in Engineering.

[58]  李敏 Li Min,et al.  Aluminum Surface-Defect Detection Based on Multi-Task Deep Learning , 2020 .

[59]  Fathi E. Abd El-Samie,et al.  A real-time approach for automatic defect detection from PCBs based on SURF features and morphological operations , 2019, Multimedia Tools and Applications.

[61]  Zhenyu He,et al.  Target-Aware Deep Tracking , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Yunhui Yan,et al.  A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects , 2013 .

[63]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[64]  Jinjiang Wang,et al.  Machine vision intelligence for product defect inspection based on deep learning and Hough transform , 2019, Journal of Manufacturing Systems.

[65]  T. Lohse,et al.  Application of deep learning methods to analysis of imaging atmospheric Cherenkov telescopes data , 2018, Astroparticle Physics.

[66]  Xiuxi Li,et al.  A Weakly Supervised Surface Defect Detection Based on Convolutional Neural Network , 2020, IEEE Access.

[67]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[68]  Da Cui,et al.  Reliability design and optimization of the planetary gear by a GA based on the DEM and Kriging model , 2020, Reliab. Eng. Syst. Saf..

[69]  Minyoung Lee,et al.  Effective automatic defect classification process based on CNN with stacking ensemble model for TFT-LCD panel , 2020, J. Intell. Manuf..

[70]  Qiang Yang,et al.  Deep learning based automatic defect identification of photovoltaic module using electroluminescence images , 2020 .

[71]  Xiaoyan Chen,et al.  A Light-Weighted CNN Model for Wafer Structural Defect Detection , 2020, IEEE Access.

[72]  Dejan Tomazevic,et al.  A Compact Convolutional Neural Network for Textured Surface Anomaly Detection , 2018, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[73]  Yuanxiang Li,et al.  Multi-robot path planning in wireless sensor networks based on jump mechanism PSO and safety gap obstacle avoidance , 2021, Future Gener. Comput. Syst..

[74]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[75]  Xiangdong Gao,et al.  Real-time Monitoring for Disk Laser Welding Based on Feature Selection and SVM , 2017 .

[76]  In-So Kweon,et al.  CBAM: Convolutional Block Attention Module , 2018, ECCV.

[77]  Freddy Lécué,et al.  On The Role of Knowledge Graphs in Explainable AI , 2020, PROFILES/SEMEX@ISWC.

[78]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[79]  Bo Chen,et al.  MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications , 2017, ArXiv.

[80]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[81]  Junliang Wang,et al.  An Unequal Deep Learning Approach for 3-D Point Cloud Segmentation , 2020, IEEE Transactions on Industrial Informatics.

[82]  Yunhui Yan,et al.  Steel Surface Defect Classification Based on Discriminant Manifold Regularized Local Descriptor , 2018, IEEE Access.

[83]  Zhenzhi He,et al.  A Hybrid Data-Driven Method for Wire Rope Surface Defect Detection , 2020, IEEE Sensors Journal.

[84]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[85]  Shibin Gao,et al.  Contact Wire Support Defect Detection Using Deep Bayesian Segmentation Neural Networks and Prior Geometric Knowledge , 2019, IEEE Access.

[86]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[87]  Luofeng Xie,et al.  FFCNN: A Deep Neural Network for Surface Defect Detection of Magnetic Tile , 2021, IEEE Transactions on Industrial Electronics.

[88]  Dawei Li,et al.  Automatic Detection and Classification of Sewer Defects via Hierarchical Deep Learning , 2019, IEEE Transactions on Automation Science and Engineering.

[89]  Nojun Kwak,et al.  Analysis on the Dropout Effect in Convolutional Neural Networks , 2016, ACCV.

[90]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[91]  Ge Zhang,et al.  Automated detection of defects with low semantic information in X-ray images based on deep learning , 2020, J. Intell. Manuf..

[92]  Meng Shi,et al.  Magnetic flux leakage image classification method for pipeline weld based on optimized convolution kernel , 2019, Neurocomputing.

[93]  Feng Wei,et al.  Research on defect detection method for high-reflective-metal surface based on high dynamic range imaging , 2020, Optik.

[94]  Qinggang Meng,et al.  An End-to-End Steel Surface Defect Detection Approach via Fusing Multiple Hierarchical Features , 2020, IEEE Transactions on Instrumentation and Measurement.

[95]  Mu Ping'an,et al.  Wafer Defects Detecting and Classifying System Based on Machine Vision , 2007, 2007 8th International Conference on Electronic Measurement and Instruments.

[96]  Xiong Luo,et al.  Detecting Defects With Support Vector Machine in Logistics Packaging Boxes for Edge Computing , 2020, IEEE Access.

[97]  Andrew H. Sung,et al.  Ranking importance of input parameters of neural networks , 1998 .

[98]  Perry P. Gao,et al.  Welding defects detection based on deep learning with multiple optical sensors during disk laser welding of thick plates , 2019, Journal of Manufacturing Systems.

[99]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[100]  Miao Qian,et al.  Yarn-dyed woven fabric density measurement method and system based on multi-directional illumination image fusion enhancement technology , 2020 .

[101]  Christoph J. Brabec,et al.  A Benchmark for Visual Identification of Defective Solar Cells in Electroluminescence Imagery , 2018 .

[102]  Filip Karlo Dosilovic,et al.  Explainable artificial intelligence: A survey , 2018, 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[103]  Takayuki Okatani,et al.  A vision-based method for crack detection in gusset plate welded joints of steel bridges using deep convolutional neural networks , 2019, Automation in Construction.

[104]  George Chryssolouris,et al.  Digital twin-driven supervised machine learning for the development of artificial intelligence applications in manufacturing , 2020, Int. J. Comput. Integr. Manuf..

[105]  H. K. Sardana,et al.  An independent component analysis based approach for frequency modulated thermal wave imaging for subsurface defect detection in steel sample , 2019, Infrared Physics & Technology.

[106]  Peng Wang,et al.  Long short-term memory for machine remaining life prediction , 2018, Journal of Manufacturing Systems.

[107]  Quoc V. Le,et al.  DropBlock: A regularization method for convolutional networks , 2018, NeurIPS.

[108]  Andrea Prati,et al.  Fashion Product Classification through Deep Learning and Computer Vision , 2019, Applied Sciences.

[109]  LiMin Fu,et al.  Rule Generation from Neural Networks , 1994, IEEE Trans. Syst. Man Cybern. Syst..

[110]  Michael Felsberg,et al.  ECO: Efficient Convolution Operators for Tracking , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[111]  Ming-Ju Wu,et al.  Wafer Map Failure Pattern Recognition and Similarity Ranking for Large-Scale Data Sets , 2015, IEEE Transactions on Semiconductor Manufacturing.

[112]  Liang Gao,et al.  An effective multi-objective discrete virus optimization algorithm for flexible job-shop scheduling problem with controllable processing times , 2017, Comput. Ind. Eng..

[113]  François Clemens,et al.  A defect classification methodology for sewer image sets with convolutional neural networks , 2019, Automation in Construction.

[114]  Bin Jiang,et al.  Automatic crack distress classification from concrete surface images using a novel deep-width network architecture , 2020, Neurocomputing.

[115]  Chang Ouk Kim,et al.  Convolutional Neural Network for Wafer Surface Defect Classification and the Detection of Unknown Defect Class , 2019, IEEE Transactions on Semiconductor Manufacturing.

[116]  Zachary C. Lipton,et al.  The mythos of model interpretability , 2018, Commun. ACM.

[117]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[118]  Xinyu Li,et al.  A semi-supervised convolutional neural network-based method for steel surface defect recognition , 2020, Robotics Comput. Integr. Manuf..

[119]  Alvaro Rodriguez-Tajes,et al.  A convolutional approach to quality monitoring for laser manufacturing , 2019, J. Intell. Manuf..

[120]  Josef Kittler,et al.  Learning Adaptive Discriminative Correlation Filters via Temporal Consistency Preserving Spatial Feature Selection for Robust Visual Object Tracking , 2018, IEEE Transactions on Image Processing.

[121]  Seokho Kang,et al.  Joint modeling of classification and regression for improving faulty wafer detection in semiconductor manufacturing , 2018, J. Intell. Manuf..

[122]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[123]  Qiyue Wang,et al.  End-to-end prediction of weld penetration: A deep learning and transfer learning based method , 2020 .

[124]  Jonathan Tompson,et al.  Efficient object localization using Convolutional Networks , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[125]  Yunhui Yan,et al.  Saliency detection for strip steel surface defects using multiple constraints and improved texture features , 2020 .

[126]  Weifeng Zhang,et al.  Real-time penetration state monitoring using convolutional neural network for laser welding of tailor rolled blanks , 2020 .

[127]  Chao Lu,et al.  A multi-objective cellular grey wolf optimizer for hybrid flowshop scheduling problem considering noise pollution , 2019, Appl. Soft Comput..

[128]  Xi Vincent Wang,et al.  A Multilevel Information Fusion-Based Deep Learning Method for Vision-Based Defect Recognition , 2020, IEEE Transactions on Instrumentation and Measurement.

[129]  Sheng-Jen Hsieh,et al.  Using active thermography to inspect pin-hole defects in anti-reflective coating with k-mean clustering , 2015 .

[130]  Xiaodong Gu,et al.  Towards dropout training for convolutional neural networks , 2015, Neural Networks.

[131]  Jing Liu,et al.  A two-stage attention aware method for train bearing shed oil inspection based on convolutional neural networks , 2020, Neurocomputing.

[132]  Usman Tariq,et al.  Application of Machine Learning to Evaluate Insulator Surface Erosion , 2020, IEEE Transactions on Instrumentation and Measurement.

[133]  Dazhong Wu,et al.  Deep learning for smart manufacturing: Methods and applications , 2018, Journal of Manufacturing Systems.

[134]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[135]  Mayorkinos Papaelias,et al.  Automated defect classification of SS304 TIG welding process using visible spectrum camera and machine learning , 2019, NDT & E International.

[136]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[137]  Zhifen Zhang,et al.  Real-time seam defect identification for Al alloys in robotic arc welding using optical spectroscopy and integrating learning , 2020 .

[138]  Yongfang Xie,et al.  A semi-supervised Laplacian extreme learning machine and feature fusion with CNN for industrial superheat identification , 2020, Neurocomputing.

[139]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[140]  Bogdan Ruszczak,et al.  Detection of power line insulators on digital images with the use of laser spots , 2019, IET Image Process..

[141]  Jian Gao,et al.  Automatic surface defect detection for mobile phone screen glass based on machine vision , 2017, Appl. Soft Comput..

[142]  Matthias Bethge,et al.  ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness , 2018, ICLR.

[143]  Haibo He,et al.  A visual long-short-term memory based integrated CNN model for fabric defect image classification , 2020, Neurocomputing.

[144]  Zhenjie Hou,et al.  Fabric defect inspection based on lattice segmentation and Gabor filtering , 2017, Neurocomputing.

[145]  Xiaoping Ma,et al.  Densely pyramidal residual network for UAV-based railway images dehazing , 2020, Neurocomputing.

[146]  Xin Lin,et al.  Image edge detection based tool condition monitoring with morphological component analysis. , 2017, ISA transactions.

[147]  Albert Y. Zomaya,et al.  An Unmanned Inspection System for Multiple Defects Detection in Photovoltaic Plants , 2020, IEEE Journal of Photovoltaics.

[148]  Geoffrey E. Hinton,et al.  Dynamic Routing Between Capsules , 2017, NIPS.

[149]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[150]  Lin Tang,et al.  Polycrystalline silicon wafer defect segmentation based on deep convolutional neural networks , 2020, Pattern Recognit. Lett..

[151]  Jinsong Bao,et al.  A Coarse-Grained Regularization Method of Convolutional Kernel for Molten Pool Defect Identification , 2020, J. Comput. Inf. Sci. Eng..