Deep heterogeneous GRU model for predictive analytics in smart manufacturing: Application to tool wear prediction

Abstract Smart manufacturing arises the growing demand for predictive analytics to forecast the deterioration and reliability of equipment. Many machine learning algorithms, especially deep learning, have been investigated for the above tasks. However, long-term prediction is still considered as a challenging issue. To address this problem, this paper presents a hybrid prediction scheme accomplished by a newly developed deep heterogeneous GRU model, along with local feature extraction. Specifically, to capture the temporal pattern hidden in the sequential input, a local feature extraction method is designed by integrating expertise knowledge into the deep learning model for enhanced feature learning. Next, an intermediate layer is designed in the deep heterogeneous GRU model structure to capture the inherent relation for long-term prediction. The proposed model is optimized by systematic feature engineering and optimal hyperparameter searching. Finally, experimental studies on tool wear test are performed to validate the superiority of the presented model over conventional approaches.

[1]  Fei Shen,et al.  Machine Health Monitoring Using Local Feature-Based Gated Recurrent Unit Networks , 2018, IEEE Transactions on Industrial Electronics.

[2]  Haidong Shao,et al.  Rolling bearing fault detection using continuous deep belief network with locally linear embedding , 2018, Comput. Ind..

[3]  Jinjiang Wang,et al.  Deep Boltzmann machine based condition prediction for smart manufacturing , 2018, Journal of Ambient Intelligence and Humanized Computing.

[4]  Dimitris Kiritsis,et al.  Deep learning for big data applications in CAD and PLM - Research review, opportunities and case study , 2018, Comput. Ind..

[5]  Robert X. Gao,et al.  Enhanced particle filter for tool wear prediction , 2015 .

[6]  Ruqiang Yan,et al.  Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks , 2017, Sensors.

[7]  Rik Van de Walle,et al.  Deep Learning for Infrared Thermal Image Based Machine Health Monitoring , 2017, IEEE/ASME Transactions on Mechatronics.

[8]  Peng Chen,et al.  Step-by-Step Fuzzy Diagnosis Method for Equipment Based on Symptom Extraction and Trivalent Logic Fuzzy Diagnosis Theory , 2018, IEEE Transactions on Fuzzy Systems.

[9]  Ha Young Kim,et al.  Forecasting the volatility of stock price index: A hybrid model integrating LSTM with multiple GARCH-type models , 2018, Expert Syst. Appl..

[10]  Sung Wook Baik,et al.  Action Recognition in Video Sequences using Deep Bi-Directional LSTM With CNN Features , 2018, IEEE Access.

[11]  Klaus-Dieter Thoben,et al.  Machine learning in manufacturing: advantages, challenges, and applications , 2016 .

[12]  Jin Cui,et al.  Multi-bearing remaining useful life collaborative prediction: A deep learning approach , 2017 .

[13]  Daniel Rueckert,et al.  Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction , 2017, IEEE Transactions on Medical Imaging.

[14]  Liang Guo,et al.  A recurrent neural network based health indicator for remaining useful life prediction of bearings , 2017, Neurocomputing.

[15]  Robert X. Gao,et al.  A deep learning-based approach to material removal rate prediction in polishing , 2017 .

[16]  Robert X. Gao,et al.  Cloud-enabled prognosis for manufacturing , 2015 .

[17]  Dong-Gyu Lee,et al.  Discriminative context learning with gated recurrent unit for group activity recognition , 2018, Pattern Recognit..

[18]  Connor Jennings,et al.  A Comparative Study on Machine Learning Algorithms for Smart Manufacturing: Tool Wear Prediction Using Random Forests , 2017 .

[19]  Yuanqing Xia,et al.  A deep Boltzmann machine and multi-grained scanning forest ensemble collaborative method and its application to industrial fault diagnosis , 2018, Comput. Ind..

[20]  Qingshan Liu,et al.  Cascaded Recurrent Neural Networks for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Dazhong Wu,et al.  Deep learning for smart manufacturing: Methods and applications , 2018, Journal of Manufacturing Systems.

[22]  Radu-Emil Precup,et al.  An overview on fault diagnosis and nature-inspired optimal control of industrial process applications , 2015, Comput. Ind..

[23]  Dragan Djurdjanovic,et al.  Feature extraction, condition monitoring, and fault modeling in semiconductor manufacturing systems , 2013, Comput. Ind..

[24]  Lixiang Duan,et al.  Deep learning enabled intelligent fault diagnosis: Overview and applications , 2018, J. Intell. Fuzzy Syst..

[25]  Tianliang Hu,et al.  Design and development of a CNC machining process knowledge base using cloud technology , 2016, The International Journal of Advanced Manufacturing Technology.

[26]  Robert X. Gao,et al.  A virtual sensing based augmented particle filter for tool condition prognosis , 2017 .

[27]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[28]  Lianhong Cai,et al.  Question detection from acoustic features using recurrent neural network with gated recurrent unit , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[29]  Hongwen He,et al.  Long Short-Term Memory Recurrent Neural Network for Remaining Useful Life Prediction of Lithium-Ion Batteries , 2018, IEEE Transactions on Vehicular Technology.

[30]  Krzysztof Jemielniak,et al.  Advanced monitoring of machining operations , 2010 .

[31]  Peter C. Y. Chen,et al.  LSTM network: a deep learning approach for short-term traffic forecast , 2017 .

[32]  Zude Zhou,et al.  Condition monitoring towards energy-efficient manufacturing: a review , 2017 .

[33]  Yibo Li,et al.  Multi-View Hierarchical Bidirectional Recurrent Neural Network for Depth Video Sequence Based Action Recognition , 2018, Int. J. Pattern Recognit. Artif. Intell..

[34]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[35]  Jaime Campos,et al.  Development in the application of ICT in condition monitoring and maintenance , 2009, Comput. Ind..

[36]  Xiang Li,et al.  Remaining useful life estimation in prognostics using deep convolution neural networks , 2018, Reliab. Eng. Syst. Saf..

[37]  Lihui Wang,et al.  Current status and advancement of cyber-physical systems in manufacturing , 2015 .

[38]  Andrew Kusiak,et al.  Smart manufacturing must embrace big data , 2017, Nature.

[39]  Peter W. Tse,et al.  Prediction of Machine Deterioration Using Vibration Based Fault Trends and Recurrent Neural Networks , 1999 .

[40]  Jayanthi Sivaswamy,et al.  RACE-Net: A Recurrent Neural Network for Biomedical Image Segmentation , 2019, IEEE Journal of Biomedical and Health Informatics.

[41]  Luis Ribeiro,et al.  Re-thinking diagnosis for future automation systems: An analysis of current diagnostic practices and their applicability in emerging IT based production paradigms , 2011, Comput. Ind..

[42]  Xiaojun Zhou,et al.  Intelligent monitoring and diagnosis of manufacturing processes using an integrated approach of KBANN and GA , 2008, Comput. Ind..

[43]  Jia Liu,et al.  Advanced recurrent network-based hybrid acoustic models for low resource speech recognition , 2018, EURASIP J. Audio Speech Music. Process..