A hybrid wrapper-filter approach to detect the source(s) of out-of-control signals in multivariate manufacturing process

With modern data-acquisition equipment and on-line computers used during production, it is now common to monitor several correlated quality characteristics simultaneously in multivariate processes. Multivariate control charts (MCC) are important tools for monitoring multivariate processes. One difficulty encountered with multivariate control charts is the identification of the variable or group of variables that cause an out-of-control signal. Expert knowledge either in combination with wrapper-based supervised classifier or a pre-filter with wrapper are the standard approaches to detect the sources of out-of-control signal. However gathering expert knowledge in source identification is costly and may introduce human error. Individual univariate control charts (UCC) and decomposition of T2 statistics are also used in many cases simultaneously to identify the sources, but these either ignore the correlations between the sources or may take more time with the increase of dimensions. The aim of this paper is to develop a source identification approach that does not need any expert-knowledge and can detect out-of-control signal in less computational complexity. We propose, a hybrid wrapper–filter based source identification approach that hybridizes a Mutual Information (MI) based Maximum Relevance (MR) filter ranking heuristic with an Artificial Neural Network (ANN) based wrapper. The Artificial Neural Network Input Gain Measurement Approximation (ANNIGMA) has been combined with MR (MR-ANNIGMA) to utilize the knowledge about the intrinsic pattern of the quality characteristics computed by the filter for directing the wrapper search process. To compute optimal ANNIGMA score, we also propose a Global MR-ANNIGMA using non-functional relationship between variables which is independent of the derivative of the objective function and has a potential to overcome the local optimization problem of ANN training. The novelty of the proposed approaches is that they combine the advantages of both filter and wrapper approaches and do not require any expert knowledge about the sources of the out-of-control signals. Heuristic score based subset generation process also reduces the search space into polynomial growth which in turns reduces computational time. The proposed approaches were tested by exhaustive experiments using both simulated and real manufacturing data and compared to existing methods including independent filter, wrapper and Multivariate EWMA (MEWMA) methods. The results indicate that the proposed approaches can identify the sources of out-of-control signals more accurately than existing approaches.

[1]  M. Ng,et al.  Informative Gene Discovery for Cancer Classification from Microarray Expression Data , 2005, 2005 IEEE Workshop on Machine Learning for Signal Processing.

[2]  Xiaojun Zhou,et al.  Identifying source(s) of out-of-control signals in multivariate manufacturing processes using selective neural network ensemble , 2009, Eng. Appl. Artif. Intell..

[3]  R. J. Mayer,et al.  Using the Taguchi paradigm for manufacturing design using simulation experiments , 1992 .

[4]  Shingo Mabu,et al.  A Global Optimization Method RasID-GA for Neural Network Training , 2008, J. Adv. Comput. Intell. Intell. Informatics.

[5]  Fu-Kwun Wang,et al.  QUALITY EVALUATION USING GEOMETRIC DISTANCE APPROACH , 1999 .

[6]  Mitra Fouladirad,et al.  A methodology for probabilistic model-based prognosis , 2013, Eur. J. Oper. Res..

[7]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[8]  W. T. Tucker,et al.  Identification of out of control quality characteristics in a multivariate manufacturing environment , 1991 .

[9]  Yoseba K. Penya,et al.  Idea: Opcode-Sequence-Based Malware Detection , 2010, ESSoS.

[10]  Juan Humberto Sossa Azuela,et al.  Evolving Neural Networks: A Comparison between Differential Evolution and Particle Swarm Optimization , 2011, ICSI.

[11]  Tamara G. Kolda,et al.  Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods , 2003, SIAM Rev..

[12]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[13]  Michael R. Lyu,et al.  Gene Selection Based on Mutual Information for the Classification of Multi-class Cancer , 2006, ICIC.

[14]  Musa Mammadov,et al.  Dynamical Systems Described by Relational Elasticities with Applications , 2005 .

[15]  Abdessamad Kobi,et al.  Fault detection and identification with a new feature selection based on mutual information , 2008 .

[16]  T. Du,et al.  Using principal component analysis in process performance for multivariate data , 2000 .

[17]  Chih-Ming Hsu,et al.  Analysis of variations in a multi-variate process using neural networks , 2003 .

[18]  J. Jackson,et al.  An Application of Multivariate Quality Control to Photographic Processing , 1957 .

[19]  F. Zorriassatine,et al.  Using novelty detection to identify abnormalities caused by mean shifts in bivariate processes , 2003 .

[20]  Lukasz A. Kurgan,et al.  CAIM discretization algorithm , 2004, IEEE Transactions on Knowledge and Data Engineering.

[21]  Igor Santos,et al.  OPEM: A Static-Dynamic Approach for Machine-Learning-Based Malware Detection , 2012, CISIS/ICEUTE/SOCO Special Sessions.

[22]  B. J. Murphy.  Selecting Out of Control Variables with the T2 Multivariate Quality Control Procedure , 1987 .

[23]  Francisco Aparisi,et al.  Interpreting the Out-of-Control Signals of Multivariate Control Charts Employing Neural Networks , 2010 .

[24]  Charles W. Champ,et al.  A multivariate exponentially weighted moving average control chart , 1992 .

[25]  Josep M. Sopena,et al.  Performing Feature Selection With Multilayer Perceptrons , 2008, IEEE Transactions on Neural Networks.

[26]  Michael R. Lyu,et al.  A hybrid particle swarm optimization-back-propagation algorithm for feedforward neural network training , 2007, Appl. Math. Comput..

[27]  Chih-Fong Tsai,et al.  Feature selection in bankruptcy prediction , 2009, Knowl. Based Syst..

[28]  Gerhard Fettweis,et al.  Base Station Placement Based on Force Fields , 2012, 2012 IEEE 75th Vehicular Technology Conference (VTC Spring).

[29]  Wenbin Wang,et al.  A simulation-based multivariate Bayesian control chart for real time condition-based maintenance of complex systems , 2012, Eur. J. Oper. Res..

[30]  David A. Bell,et al.  Axiomatic Approach to Feature Subset Selection Based on Relevance , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  James A. Rodger,et al.  Toward reducing failure risk in an integrated vehicle health maintenance system: A fuzzy multi-sensor data fusion Kalman filter approach for IVHMS , 2012, Expert Syst. Appl..

[32]  Tai-Yue Wang,et al.  Artificial neural networks to classify mean shifts from multivariate χ2 chart signals , 2004, Comput. Ind. Eng..

[33]  Lifeng Xi,et al.  A neural network ensemble-based model for on-line monitoring and diagnosis of out-of-control signals in multivariate manufacturing processes , 2009, Expert Syst. Appl..

[34]  George C. Runger,et al.  Comparison of multivariate CUSUM charts , 1990 .

[35]  Nola D. Tracy,et al.  Decomposition of T2 for Multivariate Control Chart Interpretation , 1995 .

[36]  Alexey Tsymbal,et al.  Advanced local feature selection in medical diagnostics , 2000, Proceedings 13th IEEE Symposium on Computer-Based Medical Systems. CBMS 2000.

[37]  Alberto Tesi,et al.  On the Problem of Local Minima in Backpropagation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Deborah F. Cook,et al.  Environmental statistical process control using an augmented neural network classification approach , 2006, Eur. J. Oper. Res..

[39]  Ali Maroosi,et al.  A new clustering algorithm based on hybrid global optimizationbased on a dynamical systems approach algorithm , 2010, Expert Syst. Appl..

[40]  Xiaoming Xu,et al.  A parameterless feature ranking algorithm based on MI , 2008, Neurocomputing.

[41]  Francesc J. Ferri,et al.  Comparative study of techniques for large-scale feature selection* *This work was suported by a SERC grant GR/E 97549. The first author was also supported by a FPI grant from the Spanish MEC, PF92 73546684 , 1994 .

[42]  Michel Verleysen,et al.  Feature Scoring by Mutual Information for Classification of Mass Spectra , 2006 .

[43]  Kevin W. Linderman,et al.  An integrated systems approach to process control and maintenance , 2005, Eur. J. Oper. Res..