Fault detection based on one-class deep learning for manufacturing applications limited to an imbalanced database

Abstract Despite extensive studies for the industrial applications of deep learning, its actual usage in manufacturing sites has been extremely restrained by the difficulty in obtaining sufficient industrial data, especially for production failure cases. In this study, we introduced a fault-detection module based on one-class deep learning for imbalanced industrial time-series data, which consists of three submodules, namely, time-series prediction based on deep learning, residual calculation, and one-class classification using one-class support vector machine and isolation forest. Four different networks were used for the time-series prediction: multilayer perception (MLP), residual network (ResNet), long–short-term memory (LSTM), and ResNet–LSTM, each trained with the one-class data having only the production success cases. We adopted the residuals of the deep-learning prediction as an elaborated feature for the construction of the one-class classification. We also tested the fault-detection module with the actual mass production data of a die-casting process. By adopting the features elaborated by the deep-learning time-series prediction, we showed that the total accuracy of the one-class classification significantly improved from 90.0% to 96.0%. Especially for its capability to detect production failures, the accuracy improved from 84.0% to 96.0%. The area under the receiver operating characteristics (AUROC) also improved from 87.56% to 98.96%. ResNet showed the best performance for detecting production failures, whereas ResNet–LSTM produced better results for ensuring the production success. Our results suggest that the one-class deep learning is a promising approach for extracting important features from time-series data to realize a one-class fault-detection module.

[1]  Rahat Iqbal,et al.  Fault Detection and Isolation in Industrial Processes Using Deep Learning Approaches , 2019, IEEE Transactions on Industrial Informatics.

[2]  Francisco Herrera,et al.  Weighted one-class classification for different types of minority class examples in imbalanced data , 2014, 2014 IEEE Symposium on Computational Intelligence and Data Mining (CIDM).

[3]  Ah Chung Tsoi,et al.  Discrete time recurrent neural network architectures: A unifying review , 1997, Neurocomputing.

[4]  Bin Li,et al.  Early Fault Detection of Machine Tools Based on Deep Learning and Dynamic Identification , 2019, IEEE Transactions on Industrial Electronics.

[5]  Eyad Elyan,et al.  MFC-GAN: Class-imbalanced dataset classification using Multiple Fake Class Generative Adversarial Network , 2019, Neurocomputing.

[6]  Ying Liu,et al.  Energy consumption modelling using deep learning technique — a case study of EAF , 2018 .

[7]  Sanjay Chawla,et al.  Deep Learning for Anomaly Detection: A Survey , 2019, ArXiv.

[8]  Robert X. Gao,et al.  Deep learning and its applications to machine health monitoring , 2019, Mechanical Systems and Signal Processing.

[9]  Zhenyu Liu,et al.  Petri-net-based dynamic scheduling of flexible manufacturing system via deep reinforcement learning with graph convolutional network , 2020 .

[10]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[11]  Xinggang Wang,et al.  Automated defect inspection of LED chip using deep convolutional neural network , 2019, J. Intell. Manuf..

[12]  Yang Feng,et al.  Unsupervised Anomaly Detection via Variational Auto-Encoder for Seasonal KPIs in Web Applications , 2018, WWW.

[13]  Pingyu Jiang,et al.  Combining granular computing technique with deep learning for service planning under social manufacturing contexts , 2017, Knowl. Based Syst..

[14]  Jürgen Schmidhuber,et al.  LSTM can Solve Hard Long Time Lag Problems , 1996, NIPS.

[15]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[16]  Chang Ouk Kim,et al.  A Convolutional Neural Network for Fault Classification and Diagnosis in Semiconductor Manufacturing Processes , 2017, IEEE Transactions on Semiconductor Manufacturing.

[17]  George Papandreou,et al.  Rethinking Atrous Convolution for Semantic Image Segmentation , 2017, ArXiv.

[18]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[19]  Sanjay Chawla,et al.  Anomaly Detection using One-Class Neural Networks , 2018, ArXiv.

[20]  Srikanth Kandula,et al.  Resource Management with Deep Reinforcement Learning , 2016, HotNets.

[21]  Giulio Di Gravio,et al.  Machine learning for anomaly detection and process phase classification to improve safety and maintenance activities , 2020, Journal of Manufacturing Systems.

[22]  Mahmood Fathy,et al.  Adversarially Learned One-Class Classifier for Novelty Detection , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[23]  Bernhard Schölkopf,et al.  A Primer on Kernel Methods , 2004 .

[24]  Ruqiang Yan,et al.  Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks , 2017, Sensors.

[25]  Alexander Binder,et al.  Deep One-Class Classification , 2018, ICML.

[26]  Sanjiv Kumar,et al.  On the Convergence of Adam and Beyond , 2018 .

[27]  Young Chul Lee,et al.  Migration from the traditional to the smart factory in the die-casting industry: Novel process data acquisition and fault detection based on artificial neural network , 2021 .

[28]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[29]  Yongsub Lim,et al.  RaPP: Novelty Detection with Reconstruction along Projection Pathway , 2020, ICLR.

[30]  Xin Lu,et al.  Fog Computing and Convolutional Neural Network Enabled Prognosis for Machining Process Optimization , 2019, Springer Series in Advanced Manufacturing.

[31]  Georg Langs,et al.  Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery , 2017, IPMI.

[32]  Yu Cheng,et al.  Deep Structured Energy Based Models for Anomaly Detection , 2016, ICML.

[33]  Jack Beuth,et al.  Anomaly Detection and Classification in a Laser Powder Bed Additive Manufacturing Process using a Trained Computer Vision Algorithm , 2018 .

[34]  Guanghua Xu,et al.  Health indicator construction of machinery based on end-to-end trainable convolution recurrent neural networks , 2020 .

[35]  Bo Zong,et al.  Deep Autoencoding Gaussian Mixture Model for Unsupervised Anomaly Detection , 2018, ICLR.

[36]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[37]  Bernhard Schölkopf,et al.  Support Vector Method for Novelty Detection , 1999, NIPS.

[38]  Hui Li,et al.  Computer vision and deep learning–based data anomaly detection method for structural health monitoring , 2019 .

[39]  Fernando Bação,et al.  Effective data generation for imbalanced learning using conditional generative adversarial networks , 2018, Expert Syst. Appl..

[40]  Jinjiang Wang,et al.  Machine vision intelligence for product defect inspection based on deep learning and Hough transform , 2019, Journal of Manufacturing Systems.

[41]  Burak Acar,et al.  Anomaly monitoring improves remaining useful life estimation of industrial machinery , 2020 .

[42]  Gyogwon Koo,et al.  Automated defect inspection system for metal surfaces based on deep learning and data augmentation , 2020 .

[43]  Lei Ren,et al.  Bearing remaining useful life prediction based on deep autoencoder and deep neural networks , 2018, Journal of Manufacturing Systems.

[44]  David A. Clifton,et al.  A review of novelty detection , 2014, Signal Process..