On the Application of Automated Machine Vision for Leather Defect Inspection and Grading: A Survey

Reliably and effectively detecting and classifying leather surface defects is of great importance to tanneries and industries that use leather as a major raw material such as leather footwear and handbag manufacturers. This paper presents a detailed and methodical review of the leather surface defects, their effects on leather quality grading and automated visual inspection methods for leather defect inspection. A detailed review of inspection methods based on leather defect detection using image analysis methods is presented, which are usually classified as heuristic or basic machine learning based methods. Due to the recent success of deep learning methods in various related fields, various architectures of deep learning are discussed that are tailored to image classification, detection, and segmentation. In general, visual inspection applications, where recent CNN architectures are classified, compared, and a detailed review is subsequently presented on the role of deep learning methods in leather defect detection. Finally, research guidelines are presented to fellow researchers regarding data augmentation, leather quality quantification, and simultaneous defect inspection methods, which need to be investigated in the future to make progress in this crucial area of research.

[1]  Mohammad R. Jahanshahi,et al.  NB-CNN: Deep Learning-Based Crack Detection Using Convolutional Neural Network and Naïve Bayes Data Fusion , 2018, IEEE Transactions on Industrial Electronics.

[2]  J.D. Aranda Penaranda,et al.  Inspection and measurement of leather system based on artificial vision techniques applied to the automation and waterjet cut direct application , 1994, Proceedings of IEEE International Conference on Systems, Man and Cybernetics.

[3]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Siddhartha Kumar Khaitan,et al.  Deep Convolutional Neural Networks with transfer learning for computer vision-based data-driven pavement distress detection , 2017 .

[5]  Yi Yang,et al.  Attention to Scale: Scale-Aware Semantic Image Segmentation , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Kaloyan Krastev,et al.  Leather features selection for defects' recognition using fuzzy logic , 2004, CompSysTech '04.

[7]  M.,et al.  Statistical and Structural Approaches to Texture , 2022 .

[8]  Jose A. Ventura,et al.  Automated defect inspection and classification of leather fabric , 2001, Intell. Data Anal..

[9]  Jacek Tabor,et al.  Maximum Entropy Linear Manifold for Learning Discriminative Low-Dimensional Representation , 2015, ECML/PKDD.

[10]  Kaloyan Krastev,et al.  Identification of surface leather defects , 2003, CompSysTech '03.

[11]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Sze-Teng Liong,et al.  Automatic Defect Segmentation on Leather with Deep Learning , 2019, ArXiv.

[13]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[14]  Dmitry Chetverikov,et al.  Knowledge-based line-correction rules in a machine-vision system for the leather industry , 1991 .

[15]  João Luís Sobral,et al.  Leather Inspection Based on Wavelets , 2005, IbPRIA.

[16]  Giovanni Tantussi,et al.  Design and Development of a Vision Based Leather Trimming Machine , 2002 .

[17]  George Papandreou,et al.  Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation , 2018, ECCV.

[18]  Hemerson Pistori,et al.  Attributes Reduction Applied to Leather Defects Classification , 2010, 2010 23rd SIBGRAPI Conference on Graphics, Patterns and Images.

[19]  Hemerson Pistori,et al.  Defect detection in raw hide and wet blue leather , 2006, CompIMAGE.

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Marco Mora,et al.  A New Approach for Wet Blue Leather Defect Segmentation , 2011, CIARP.

[22]  Malathy Jawahar,et al.  Machine Vision Inspection System for Detection of Leather Surface Defects , 2019 .

[23]  C. Yeh,et al.  A reference standard of defect compensation for leather transactions , 2005 .

[24]  Wen Wang,et al.  Automatic Visual Inspection for Leather Manufacture , 2006 .

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  Giovanni Attolico,et al.  Leather inspection by oriented texture analysis with a morphological approach , 1997, Proceedings of International Conference on Image Processing.

[27]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[28]  Yundong Li,et al.  Deformable Patterned Fabric Defect Detection With Fisher Criterion-Based Deep Learning , 2017, IEEE Transactions on Automation Science and Engineering.

[29]  Pedro Pedrosa Rebouças Filho,et al.  Goat Leather Quality Classification Using Computer Vision and Machine Learning , 2018, 2018 International Joint Conference on Neural Networks (IJCNN).

[30]  Jonathan T. Barron,et al.  Semantic Image Segmentation with Task-Specific Edge Detection Using CNNs and a Discriminatively Trained Domain Transform , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  K. Vani,et al.  Leather texture classification using wavelet feature extraction technique , 2014, 2014 IEEE International Conference on Computational Intelligence and Computing Research.

[32]  Wolfgang Poelzleitner,et al.  Automatic inspection of leather surfaces , 1994, Other Conferences.

[33]  Kay Chen Tan,et al.  A Generic Deep-Learning-Based Approach for Automated Surface Inspection , 2018, IEEE Transactions on Cybernetics.

[34]  Tzu Hsin Kuo,et al.  Multicamera Fusion-Based Leather Defects Marking System , 2013 .

[35]  Rama Chellappa,et al.  Deep Multitask Learning for Railway Track Inspection , 2015, IEEE Transactions on Intelligent Transportation Systems.

[36]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[37]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Yichuan Tang,et al.  Deep Learning using Linear Support Vector Machines , 2013, 1306.0239.

[39]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[40]  Giovanni Attolico,et al.  Defect Detection on Leather by Oriented Singularities , 1997, CAIP.

[41]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[42]  Giovanni Attolico,et al.  Automated system for detection and classification of leather defects , 1996 .

[43]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Jussi Paakkari,et al.  Machine-vision-guided waterjet cutting , 1999, Industrial Lasers and Inspection.

[45]  Jürgen Schmidhuber,et al.  Steel defect classification with Max-Pooling Convolutional Neural Networks , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[46]  Bart De Schutter,et al.  Deep convolutional neural networks for detection of rail surface defects , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[47]  Hemerson Pistori,et al.  SVM with Stochastic Parameter Selection for Bovine Leather Defect Classification , 2007, PSIVT.

[48]  A.F.L. Serafim.  Multiresolution pyramids for segmentation of natural images based on autoregressive models: application to calf leather classification , 1991, Proceedings IECON '91: 1991 International Conference on Industrial Electronics, Control and Instrumentation.

[49]  Ajay Kumar,et al.  Defect detection in textured materials using Gabor filters , 2000, Conference Record of the 2000 IEEE Industry Applications Conference. Thirty-Fifth IAS Annual Meeting and World Conference on Industrial Applications of Electrical Energy (Cat. No.00CH37129).

[50]  Wojciech Czarnecki,et al.  On Loss Functions for Deep Neural Networks in Classification , 2017, ArXiv.

[51]  Zhigang Liu,et al.  Automatic Defect Detection of Fasteners on the Catenary Support Device Using Deep Convolutional Neural Network , 2018, IEEE Transactions on Instrumentation and Measurement.

[52]  Koen E. A. van de Sande,et al.  Selective Search for Object Recognition , 2013, International Journal of Computer Vision.

[53]  Bernd Scholz-Reiter,et al.  Design of deep convolutional neural network architectures for automated feature extraction in industrial inspection , 2016 .

[54]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[55]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Guojun Wen,et al.  Automatic Fabric Defect Detection with a Multi-Scale Convolutional Denoising Autoencoder Network Model , 2018, Sensors.

[57]  Robert J. Thomas,et al.  Comparison of deep convolutional neural networks and edge detectors for image-based crack detection in concrete , 2018, Construction and Building Materials.

[58]  A.F. Limas-Serafim.  Natural images segmentation for patterns recognition using edges pyramids and its application to the leather defects , 1993, Proceedings of IECON '93 - 19th Annual Conference of IEEE Industrial Electronics.

[59]  前田 俊二,et al.  Single Shot MultiBox DetectorとOptical Flowを組み合わせた逆走車両検知手法の検討 , 2018 .

[60]  Der-Baau Perng,et al.  Establishing a Demerit Count Reference Standard for the Classification and Grading of Leather Hides , 2001 .

[61]  Li Jian,et al.  Research on inspection and classification of leather surface defects based on neural network and decision tree , 2010, 2010 International Conference On Computer Design and Applications.

[62]  Kincho H. Law,et al.  Automatic localization of casting defects with convolutional neural networks , 2017, 2017 IEEE International Conference on Big Data (Big Data).

[63]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[64]  Huu-Cuong Nguyen,et al.  Vision-based Inspection System for Leather Surface Defect Detection and Classification , 2018, 2018 5th NAFOSTED Conference on Information and Computer Science (NICS).

[65]  Iasonas Kokkinos,et al.  Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs , 2014, ICLR.

[66]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[67]  Xindong Wu,et al.  Object Detection With Deep Learning: A Review , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[68]  Grzegorz Psuj.  Multi-Sensor Data Integration Using Deep Learning for Characterization of Defects in Steel Elements † , 2018, Sensors.

[69]  Dazhong Ma,et al.  Injurious or Noninjurious Defect Identification From MFL Images in Pipeline Inspection Using Convolutional Neural Network , 2017, IEEE Transactions on Instrumentation and Measurement.

[70]  Xiaodong Gu,et al.  Fully Convolutional Networks for Surface Defect Inspection in Industrial Environment , 2017, ICVS.

[71]  Rohit Sharma,et al.  An integrated machine vision based system for solving the nonconvex cutting stock problem using genetic algorithms , 1999 .

[72]  Murali Krishna Kasi,et al.  Identification of leather defects using an autoadaptive edge detection image processing algorithm , 2014, 2014 International Conference on High Performance Computing and Applications (ICHPCA).