Evolutionary LSTM-FCN networks for pattern classification in industrial processes

Abstract The Industry 4.0 revolution allows gathering big amounts of data that are used to train and deploy Artificial Intelligence algorithms to solve complex industrial problems, optimally and automatically. From those, Long-Short Term Memory Fully Convolutional Network (LSTM-FCN) networks are gaining a lot of attention over the last decade due to their capability of successfully modeling nonlinear feature interactions. However, they have not been yet fully applied for pattern classification tasks in time series data within the digital industry. In this paper, a novel approach based on an evolutionary algorithm for optimizing the networks hyperparameters and on the resulting deep learning model for pattern classification is proposed. In order to demonstrate the applicability of this method, a test scenario that involves a process related to blind fastener installation in the aeronautical industry is provided. The results achieved with the proposed approach are compared with shallow models and it is demonstrated that the proposed method obtains better results with an accuracy value of 95%.

[1]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[2]  Basilio Sierra,et al.  Deep evolutionary modeling of condition monitoring data in marine propulsion systems , 2018, Soft Comput..

[3]  Gary G. Yen,et al.  Particle swarm optimization of deep neural networks architectures for image classification , 2019, Swarm Evol. Comput..

[4]  Dazhong Wu,et al.  Deep learning for smart manufacturing: Methods and applications , 2018, Journal of Manufacturing Systems.

[5]  Mojtaba Ahmadieh Khanesar,et al.  An aerial robot for rice farm quality inspection with type-2 fuzzy neural networks tuned by particle swarm optimization-sliding mode control hybrid algorithm , 2017, Swarm Evol. Comput..

[6]  Mahamad Nabab Alam,et al.  A novel differential particle swarm optimization for parameter selection of support vector machines for monitoring metal-oxide surge arrester conditions , 2018, Swarm Evol. Comput..

[7]  Risto Miikkulainen,et al.  Evolving Neural Networks through Augmenting Topologies , 2002, Evolutionary Computation.

[8]  Yulong Wang,et al.  cPSO-CNN: An efficient PSO-based algorithm for fine-tuning hyper-parameters of convolutional neural networks , 2019, Swarm Evol. Comput..

[9]  Jia-Jun Wang,et al.  Parameter optimization and speed control of switched reluctance motor based on evolutionary computation methods , 2017, Swarm Evol. Comput..

[10]  Javier Del Ser,et al.  Data fusion and machine learning for industrial prognosis: Trends and perspectives towards Industry 4.0 , 2019, Inf. Fusion.

[11]  Dilip Kumar Pratihar,et al.  Tuning of neural networks using particle swarm optimization to model MIG welding process , 2011, Swarm Evol. Comput..

[12]  Min-Xia Zhang,et al.  Shallow and deep neural network training by water wave optimization , 2019, Swarm Evol. Comput..

[13]  Li Deng,et al.  A tutorial survey of architectures, algorithms, and applications for deep learning , 2014, APSIPA Transactions on Signal and Information Processing.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Nguyen Lu Dang Khoa,et al.  Kernel-based support vector machines for automated health status assessment in monitoring sensor data , 2018 .

[16]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[17]  Junliang Liu,et al.  Convolutional neural networks for time series classification , 2017 .

[18]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[19]  Jian Pei,et al.  A brief survey on sequence classification , 2010, SKDD.

[20]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[21]  Darrell Whitley,et al.  A genetic algorithm tutorial , 1994, Statistics and Computing.

[22]  Yi Zheng,et al.  Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks , 2014, WAIM.

[23]  Tak-Chung Fu,et al.  A review on time series data mining , 2011, Eng. Appl. Artif. Intell..

[24]  Dario Floreano,et al.  Neuroevolution: from architectures to learning , 2008, Evol. Intell..

[25]  Masoud Salehi Borujeni,et al.  Time series analysis and short-term forecasting of solar irradiation, a new hybrid approach , 2017, Swarm Evol. Comput..

[26]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .