Introduction of a time series machine learning methodology for the application in a production system

Abstract Machine learning methods are considered a promising approach for improving operations and processes in manufacturing. However, the application of machine learning often requires the expertise of a data scientist combined with thorough knowledge of the manufacturing processes. Small and medium-sized companies that specialize in certain high value-added, variant rich production processes often lack an in-house data scientist and therefore miss out on generating a deeper data-driven insight from their production data streams. This paper proposes a three-step machine learning methodology to empower process experts with limited knowledge in machine learning: 1) data exploration through clustering, 2) representation of the production systems behaviour through specially structured neural networks and 3) querying this representation through evolutionary algorithms to achieve decision support through online optimization or scenario simulation. The chosen algorithms focus on parameter-light, well-established, general use algorithms in order to lower knowledge requirements for their application.

[1]  Amrik S. Sohal,et al.  Assessing manufacturing plant competitiveness - An empirical field study , 2001 .

[2]  Donald E. Grierson,et al.  Comparison among five evolutionary-based optimization algorithms , 2005, Adv. Eng. Informatics.

[3]  Malte Brettel,et al.  How Virtualization, Decentralization and Network Building Change the Manufacturing Landscape: An Industry 4.0 Perspective , 2014 .

[4]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[5]  Roger Evans,et al.  Identifying the UK's manufacturing challenges as a benchmark for future growth , 2012 .

[6]  Carlos Agón,et al.  Time-series data mining , 2012, CSUR.

[7]  T. Bastogne,et al.  Application of subspace methods to the identification of a winding process , 1997, 1997 European Control Conference (ECC).

[8]  Mitsuo Gen,et al.  Genetic algorithms and engineering design , 1997 .

[9]  Sze-jung Wu,et al.  A Neural Network Integrated Decision Support System for Condition-Based Optimal Predictive Maintenance Policy , 2007, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[10]  Tsuyoshi Idé Why Does Subsequence Time-Series Clustering Produce Sine Waves? , 2006, PKDD.

[11]  László Monostori,et al.  ScienceDirect Variety Management in Manufacturing . Proceedings of the 47 th CIRP Conference on Manufacturing Systems Cyber-physical production systems : Roots , expectations and R & D challenges , 2014 .

[12]  Ronald L. Rivest,et al.  Training a 3-node neural network is NP-complete , 1988, COLT '88.

[13]  René Hofmann,et al.  Comparison of a physical and a data-driven model of a Packed Bed Regenerator for industrial applications , 2019, Journal of Energy Storage.

[14]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[15]  Li Da Xu,et al.  Industry 4.0: state of the art and future trends , 2018, Int. J. Prod. Res..

[16]  Lei Shu,et al.  Smart Factory of Industry 4.0: Key Technologies, Application Case, and Challenges , 2018, IEEE Access.

[17]  Pedro Paulo Balestrassi,et al.  Design of experiments on neural network's training for nonlinear time series forecasting , 2009, Neurocomputing.

[18]  Arthur L. Samuel,et al.  Some Studies in Machine Learning Using the Game of Checkers , 1967, IBM J. Res. Dev..

[19]  Hans-Peter Kriegel,et al.  OPTICS: ordering points to identify the clustering structure , 1999, SIGMOD '99.

[20]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[21]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[22]  Hans-Paul Schwefel,et al.  Numerical Optimization of Computer Models , 1982 .

[23]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[24]  Tung-Kuan Liu,et al.  Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm , 2006, IEEE Trans. Neural Networks.

[25]  Eamonn J. Keogh,et al.  Probabilistic discovery of time series motifs , 2003, KDD '03.

[26]  Rüdiger Wirth,et al.  CRISP-DM: Towards a Standard Process Model for Data Mining , 2000 .

[27]  Bilal Ahmad,et al.  Complexity in manufacturing systems and its measures: a literature review , 2018 .

[28]  T. Caliński,et al.  A dendrite method for cluster analysis , 1974 .

[29]  Michael C. Fu,et al.  Optimization via simulation: A review , 1994, Ann. Oper. Res..

[30]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Yolanda Carson,et al.  Simulation optimization: methods and applications , 1997, WSC '97.

[32]  Eugen Diaconescu,et al.  The use of NARX neural networks to predict chaotic time series , 2008 .

[33]  Mohamed S. Kamel,et al.  Modular neural networks: a survey. , 1999, International journal of neural systems.

[34]  Hoa Dinh Nguyen A data-driven framework for remaining useful life estimation , 2017 .

[35]  Agusmian P. Ompusunggu,et al.  A practical approach to combine data mining and prognostics for improved predictive maintenance , 2009 .

[36]  Mohammed Amer,et al.  A review of modularization techniques in artificial neural networks , 2019, Artificial Intelligence Review.

[37]  Nina F. Thornhill,et al.  Using process topology in plant-wide control loop performance assessment , 2006, Comput. Chem. Eng..

[38]  Neil Davey,et al.  Time Series Prediction and Neural Networks , 2001, J. Intell. Robotic Syst..

[39]  Thomas Bäck,et al.  An Overview of Evolutionary Algorithms for Parameter Optimization , 1993, Evolutionary Computation.

[40]  Chuen-Sheng Cheng,et al.  Identifying the source of variance shifts in the multivariate process using neural networks and support vector machines , 2008, Expert Syst. Appl..

[41]  H. Pierreval,et al.  Using evolutionary algorithms and simulation for the optimization of manufacturing systems , 1997 .

[42]  Kalyanmoy Deb,et al.  Messy Genetic Algorithms: Motivation, Analysis, and First Results , 1989, Complex Syst..

[43]  Klaus-Dieter Thoben,et al.  An approach to monitoring quality in manufacturing using supervised machine learning on product state data , 2013, Journal of Intelligent Manufacturing.

[44]  Rudy Setiono,et al.  Feedforward Neural Network Construction Using Cross Validation , 2001, Neural Computation.

[45]  Kishan G. Mehrotra,et al.  Efficient classification for multiclass problems using modular neural networks , 1995, IEEE Trans. Neural Networks.

[46]  Peter Tiño,et al.  Learning long-term dependencies in NARX recurrent neural networks , 1996, IEEE Trans. Neural Networks.

[47]  Detlef Gerhard Product Lifecycle Management Challenges of CPPS , 2017, Multi-Disciplinary Engineering for Cyber-Physical Production Systems.

[48]  Eamonn J. Keogh,et al.  Experimental comparison of representation methods and distance measures for time series data , 2010, Data Mining and Knowledge Discovery.

[49]  Guilherme De A. Barreto,et al.  Long-term time series prediction with the NARX network: An empirical evaluation , 2008, Neurocomputing.

[50]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[51]  Hava T. Siegelmann,et al.  On the complexity of training neural networks with continuous activation functions , 1995, IEEE Trans. Neural Networks.

[52]  Rajkumar Roy,et al.  Evolutionary computing in manufacturing industry: an overview of recent applications , 2005, Appl. Soft Comput..

[53]  Detlef Gerhard,et al.  Comparison of Time Series Clustering Algorithms for Machine State Detection , 2020 .

[54]  Yu-Chi Ho,et al.  An Explanation of Ordinal Optimization: Soft Computing for hard Problems , 1999, Inf. Sci..

[55]  Robert R. Sokal,et al.  A statistical method for evaluating systematic relationships , 1958 .

[56]  Kishan G. Mehrotra,et al.  Forecasting the behavior of multivariate time series using neural networks , 1992, Neural Networks.

[57]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[58]  I A Basheer,et al.  Artificial neural networks: fundamentals, computing, design, and application. , 2000, Journal of microbiological methods.

[59]  Magali R. G. Meireles,et al.  A comprehensive review for industrial applicability of artificial neural networks , 2003, IEEE Trans. Ind. Electron..

[60]  James Tannock,et al.  The optimisation of neural network parameters using Taguchi’s design of experiments approach: an application in manufacturing process modelling , 2005, Neural Computing & Applications.

[61]  Wimalin Laosiritaworn Artificial Neural Networks Parameters Optimization with Design of Experiments: An Application in Ferromagnetic Materials Modeling , 2009 .

[62]  Inneke Van Nieuwenhuyse,et al.  Simulation optimization in inventory replenishment: a classification , 2015 .