Machine Learning Approaches for Failure Type Detection and Predictive Maintenance

With an increasing number of embedded sensing computer systems set up in production plants, machines, cars, etc., there are new possibilities to monitor and log the data from such systems. This development makes it possible to detect anomalies and predict the failures that affect maintenance plans. This thesis divides the field of failure type detection and predictive maintenance into subsections that focus on its realization by a machine learning technique, where each area of failure type detection and predictive maintenance explains and summarizes the most relevant research results in recent years. Each technique introduced is considered from a failure type detection and predictive maintenance perspective, highlighting its assets and drawbacks. An overview in tabular form shows the direction of the current research activities. It will be shown that many issues in the area of feature extraction, feature selection, and data labeling have not been researched so far. Zusammenfassung Die zunehmende Anzahl eingebetteter Sensorsysteme, die in Produktionsanlagen, Maschinen, Autos, usw. implementiert werden, eröffnet neue Möglichkeiten, um ein solches System zu überwachen und Objektdaten zu erfassen. Diese Entwicklung ermöglicht es, Anomalien zu detektieren und Fehler, die wiederum Auswirkungen auf Wartungspläne haben, vor dem eigentlichen Auftreten zu erkennen. Diese Arbeit teilt das Forschungsfeld der Fehlertypenkennung und prädiktiven Wartung in Teilgruppen auf und fokussiert sich auf Umsetzungen mithilfe maschineller Lernverfahren. Jeder Teilbereich der Fehlertypenkennung und prädiktiven Wartung wird erklärt und die Forschungsergebnisse der letzten Jahre in den einzelnen Bereichen zusammengefasst. Jede vorgestellte Technik wird mit deren Vorund Nachteile für die Fehlertypenkennung und prädiktiven Wartung betrachtet. Ein tabellarischer Überblick zeigt die aktuelle Forschungsrichtung der Teilbereiche. Weiter werden Problemstellungen in den Bereichen Feature-Extraction, Feature-Selection und Data-Labeling aufgezeigt, die bislang noch nicht untersucht worden sind. Acknowledgment At first, I would like to thank my thesis advisors, Prof. Johannes Fürnkranz and Dr. Frederik Janssen, who accepted my thesis proposal and made this work possible by offering their advice and support. Furthermore, I would like to express my gratitude to Dr. Immanuel Schweizer for the considerable amount of time that he invested in answering my early questions, as well as for the interesting discussions and organizational support. Moreover, I would like to thank Dr. Andrei Tolstikov, Dipl. Inf. Sebastian Kauschke, Dr. Axel Schulz and Simon Hofmann for their technical support and efforts. Finally, I would like to thank my wife Kerstin and my children, Paul Maximilian and Hendrick Lennard, as well as my parents, who have always stood by me and dealt with my absence from many family occasions with a smile and never-ending support.

[1]  Kenneth A. Loparo,et al.  Physically based diagnosis and prognosis of cracked rotor shafts , 2002, SPIE Defense + Commercial Sensing.

[2]  Igor Loboda,et al.  A More Realistic Scheme of Deviation Error Representation for Gas Turbine Diagnostics , 2013 .

[3]  Feng Ding,et al.  Application of support vector machine for equipment reliability forecasting , 2008, 2008 6th IEEE International Conference on Industrial Informatics.

[4]  Enrico Sciubba,et al.  Automatic diagnostics and prognostics of energy conversion processes via knowledge-based systems , 2004 .

[5]  Jorge F. Silva,et al.  Particle-Filtering-Based Prognosis Framework for Energy Storage Devices With a Statistical Characterization of State-of-Health Regeneration Phenomena , 2013, IEEE Transactions on Instrumentation and Measurement.

[6]  Zhen He,et al.  Online monitoring and fault identification of mean shifts in bivariate processes using decision tree learning techniques , 2013, J. Intell. Manuf..

[7]  Goran Kvascev,et al.  Sensor fault detection and isolation in a thermal power plant steam separator , 2013 .

[8]  H. Weaver Applications of Discrete and Continuous Fourier Analysis , 1983 .

[9]  Nan Bai,et al.  Wind Turbine Gearbox Condition Monitoring with AAKR and Moving Window Statistic Methods , 2011 .

[10]  Madalena Costa,et al.  Multiscale entropy analysis of biological signals. , 2005, Physical review. E, Statistical, nonlinear, and soft matter physics.

[11]  J. Ross Quinlan,et al.  Improved Use of Continuous Attributes in C4.5 , 1996, J. Artif. Intell. Res..

[12]  Yinyu Ye,et al.  Approximating Global Quadratic Optimization with Convex Quadratic Constraints , 1999, J. Glob. Optim..

[13]  Zhi-Hua Zhou,et al.  Ensemble Methods: Foundations and Algorithms , 2012 .

[14]  Donald F. Specht,et al.  A general regression neural network , 1991, IEEE Trans. Neural Networks.

[15]  Donghua Zhou,et al.  A Wiener-process-based degradation model with a recursive filter algorithm for remaining useful life estimation , 2013 .

[16]  Jian-Bo Yang,et al.  Uncertain Nonlinear System Modeling and Identification Using Belief Rule-Based Systems , 2013, IUKM.

[17]  Nagi Gebraeel,et al.  Predictive Maintenance Management Using Sensor-Based Degradation Models , 2009, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[18]  Wei-Ying Ma,et al.  Improving text classification using local latent semantic indexing , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[19]  I. Daubechies Orthonormal bases of compactly supported wavelets , 1988 .

[20]  J. Richardson,et al.  A new, efficient structure for the short-time Fourier transform, with an application in code-division sonar imaging , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[21]  Anne-Marie Kermarrec,et al.  The many faces of publish/subscribe , 2003, CSUR.

[22]  Carey Bunks,et al.  CONDITION-BASED MAINTENANCE OF MACHINES USING HIDDEN MARKOV MODELS , 2000 .

[23]  Gerald Tesauro,et al.  TD-Gammon, a Self-Teaching Backgammon Program, Achieves Master-Level Play , 1994, Neural Computation.

[24]  Hamid Reza Karimi,et al.  Vibration analysis for bearing fault detection and classification using an intelligent filter , 2014 .

[25]  Rolf Isermann,et al.  Identification of Dynamic Systems: An Introduction with Applications , 2010 .

[26]  Swagatam Das,et al.  Multi-sensor data fusion using support vector machine for motor fault detection , 2012, Inf. Sci..

[27]  Quan Pan,et al.  Random Decision Forests for Object Detection , 2014 .

[28]  Jens Myrup Pedersen,et al.  A method for classification of network traffic based on C5.0 Machine Learning Algorithm , 2012, 2012 International Conference on Computing, Networking and Communications (ICNC).

[29]  Xuejun Li,et al.  A quantitative estimation technique for welding quality using local mean decomposition and support vector machine , 2016, J. Intell. Manuf..

[30]  Jian-Da Wu,et al.  An expert system for fault diagnosis in internal combustion engines using wavelet packet transform and neural network , 2009, Expert Syst. Appl..

[31]  Kazuo Tanaka,et al.  Stability analysis and design of fuzzy control systems , 1992 .

[32]  Daming Lin,et al.  A review on machinery diagnostics and prognostics implementing condition-based maintenance , 2006 .

[33]  Yaguo Lei,et al.  A new approach to intelligent fault diagnosis of rotating machinery , 2008, Expert Syst. Appl..

[34]  Ruoyu Li,et al.  Split torque type gearbox fault detection using acoustic emission and vibration sensors , 2010, 2010 International Conference on Networking, Sensing and Control (ICNSC).

[35]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[36]  Yi Wang,et al.  Fault diagnosis and prognosis using wavelet packet decomposition, Fourier transform and artificial neural network , 2013, J. Intell. Manuf..

[37]  J. Richman,et al.  Physiological time-series analysis using approximate entropy and sample entropy. , 2000, American journal of physiology. Heart and circulatory physiology.

[38]  P. Danielsson Euclidean distance mapping , 1980 .

[39]  Jyh-Shing Roger Jang,et al.  ANFIS: adaptive-network-based fuzzy inference system , 1993, IEEE Trans. Syst. Man Cybern..

[40]  P. Welch The use of fast Fourier transform for the estimation of power spectra: A method based on time averaging over short, modified periodograms , 1967 .

[41]  William W. S. Wei,et al.  Time series analysis - univariate and multivariate methods , 1989 .

[42]  Wen-An Yang,et al.  Simultaneous monitoring of mean vector and covariance matrix shifts in bivariate manufacturing processes using hybrid ensemble learning-based model , 2016, J. Intell. Manuf..

[43]  Laine Mears,et al.  Condition based maintenance-systems integration and intelligence using Bayesian classification and sensor fusion , 2015, J. Intell. Manuf..

[44]  J. Ackermann,et al.  Robust Control: The Parameter Space Approach , 2012 .

[45]  Andrzej Skowron,et al.  Rough set methods in feature selection and recognition , 2003, Pattern Recognit. Lett..

[46]  C. Micchelli,et al.  Approximation by superposition of sigmoidal and radial basis functions , 1992 .

[47]  Sylvain Létourneau,et al.  Developing Data Mining-Based Prognostic Models for CF-18 Aircraft , 2011 .

[48]  Noureddine Zerhouni,et al.  A Data-Driven Failure Prognostics Method Based on Mixture of Gaussians Hidden Markov Models , 2012, IEEE Transactions on Reliability.

[49]  T. Y. Wu,et al.  Characterization of gear faults in variable rotating speed using Hilbert-Huang Transform and instantaneous dimensionless frequency normalization , 2012 .

[50]  Long Zhang,et al.  Bearing fault diagnosis using multi-scale entropy and adaptive neuro-fuzzy inference , 2010, Expert Syst. Appl..

[51]  David He,et al.  Hidden semi-Markov model-based methodology for multi-sensor equipment health diagnosis and prognosis , 2007, Eur. J. Oper. Res..

[52]  Alejandro P. Buchmann,et al.  Complex Event Processing , 2009, it Inf. Technol..

[53]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[54]  Sylvie Galichet,et al.  Data-Driven Prognosis Applied to Complex Vacuum Pumping Systems , 2010, IEA/AIE.

[55]  Roberto Baldoni,et al.  The evolution of publish/subscribe communication systems , 2003 .

[56]  Xiang Li,et al.  A two-stage equipment predictive maintenance framework for high-performance manufacturing systems , 2012, 2012 7th IEEE Conference on Industrial Electronics and Applications (ICIEA).

[57]  Lennart Ljung,et al.  Robust control of identified models with mixed parametric and non-parametric uncertainties , 2001, 2001 European Control Conference (ECC).

[58]  Chris Van Hoof,et al.  The Best Materials for Tiny, Clever Sensors , 2004, Science.

[59]  Matthias Nussbaum,et al.  Advanced Digital Signal Processing And Noise Reduction , 2016 .

[60]  R. Sharpley,et al.  Analysis of the Intrinsic Mode Functions , 2006 .

[61]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[62]  Hashem M. Hashemian,et al.  State-of-the-Art Predictive Maintenance Techniques* , 2011, IEEE Transactions on Instrumentation and Measurement.

[64]  Long-Sheng Chen,et al.  Using SVM based method for equipment fault detection in a thermal power plant , 2011, Comput. Ind..

[65]  Roman W. Swiniarski,et al.  Rough sets as a front end of neural-networks texture classifiers , 2001, Neurocomputing.

[66]  Shunzheng Yu,et al.  Hidden semi-Markov models , 2010, Artif. Intell..

[67]  Kai Goebel,et al.  A Survey of Artificial Intelligence for Prognostics , 2007, AAAI Fall Symposium: Artificial Intelligence for Prognostics.

[68]  Eric Duviella,et al.  Advanced Pattern Recognition Approach for Fault Diagnosis of Wind Turbines , 2013, 2013 12th International Conference on Machine Learning and Applications.

[69]  Jian-Da Wu,et al.  An engine fault diagnosis system using intake manifold pressure signal and Wigner-Ville distribution technique , 2011, Expert Syst. Appl..

[70]  Ming Li Fractal Time Series—A Tutorial Review , 2010 .

[71]  Li Tan,et al.  Digital Signal Processing: Fundamentals and Applications , 2013 .

[72]  Shubha Kadambe,et al.  A comparison of the existence of 'cross terms' in the Wigner distribution and the squared magnitude of the wavelet transform and the short-time Fourier transform , 1992, IEEE Trans. Signal Process..

[73]  Uwe Kiencke,et al.  Signalverarbeitung: Zeit-Frequenz-Analyse und Schätzverfahren , 2008 .

[74]  Lambros Ekonomou,et al.  Design of artificial neural network models for the prediction of the Hellenic energy consumption , 2010, 10th Symposium on Neural Network Applications in Electrical Engineering.

[75]  Howard Austerlitz,et al.  Data Acquisition Techniques Using PCs , 1991 .

[76]  K. L. Butler An expert system based framework for an incipient failure detection and predictive maintenance system , 1996, Proceedings of International Conference on Intelligent System Application to Power Systems.

[77]  Slawomir Nowaczyk,et al.  Towards a Machine Learning Algorithm for Predicting Truck Compressor Failures Using Logged Vehicle Data , 2013, SCAI.

[78]  Zhenyuan Zhong,et al.  Fault diagnosis for diesel valve trains based on time–frequency images , 2008 .

[79]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[80]  Donghua Zhou,et al.  Remaining useful life estimation - A review on the statistical data driven approaches , 2011, Eur. J. Oper. Res..

[81]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[82]  Carl Frélicot A fuzzy-based pronostic adaptive system , 1996 .

[83]  Thomas P. Trappenberg,et al.  Fundamentals of Computational Neuroscience , 2002 .

[84]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[85]  Ying Peng,et al.  Current status of machine prognostics in condition-based maintenance: a review , 2010 .

[86]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[87]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[88]  Jian-Da Wu,et al.  A self-adaptive data analysis for fault diagnosis of an automotive air-conditioner blower , 2011, Expert Syst. Appl..

[89]  Charu C. Aggarwal,et al.  Managing and Mining Sensor Data , 2013, Springer US.

[90]  Chang-Hua Hu,et al.  Hidden Behavior Prediction of Complex Systems Based on Hybrid Information , 2013, IEEE Transactions on Cybernetics.

[91]  L. Cohen,et al.  Time-frequency distributions-a review , 1989, Proc. IEEE.

[92]  D. Dickey,et al.  Testing for unit roots in autoregressive-moving average models of unknown order , 1984 .

[93]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[94]  Wei Zhou,et al.  Autoregressive coefficient-invariant control chart pattern recognition in autocorrelated manufacturing processes using neural network ensemble , 2013, Journal of Intelligent Manufacturing.

[95]  Eric Bechhoefer,et al.  A Review of Time Synchronous Average Algorithms , 2009 .

[96]  Martin Vetterli,et al.  Fast Fourier transforms: a tutorial review and a state of the art , 1990 .

[97]  Zhigang Tian,et al.  An artificial neural network method for remaining useful life prediction of equipment subject to condition monitoring , 2012, J. Intell. Manuf..

[98]  Hooshang Jazayeri-Rad,et al.  Comparing the Fault Diagnosis Performances of Single Neural Networks and Two Ensemble Neural Networks Based on the Boosting Methods , 2014 .

[99]  Buyue Qian,et al.  Improving rail network velocity: A machine learning approach to predictive maintenance , 2014 .

[100]  Enrico Zio,et al.  Failure and reliability prediction by support vector machines regression of time series data , 2011, Reliab. Eng. Syst. Saf..

[101]  Donald F. Specht,et al.  Probabilistic neural networks , 1990, Neural Networks.

[102]  Chun-Chieh Wang,et al.  Applications of fault diagnosis in rotating machinery by using time series analysis with neural network , 2010, Expert Syst. Appl..

[103]  Charu C. Aggarwal,et al.  Feature Selection for Classification: A Review , 2014, Data Classification: Algorithms and Applications.

[104]  Charu C. Aggarwal,et al.  On the Surprising Behavior of Distance Metrics in High Dimensional Spaces , 2001, ICDT.

[105]  Richard C.M. Yam,et al.  An Integrated Maintenance Management System for an Advanced Manufacturing Company , 2001 .

[106]  Soon Heung Chang,et al.  Development of an on-line fuzzy expert system for integrated alarm processing in nuclear power plants , 1995 .

[107]  Sankalita Saha,et al.  Metrics for Offline Evaluation of Prognostic Performance , 2021, International Journal of Prognostics and Health Management.

[108]  Otto Föllinger Laplace- und Fourier-Transformation , 1980 .

[109]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.