A robust unsupervised consensus control chart pattern recognition framework

We propose an unsupervised consensus approach for the control chart pattern recognition problem.We provide computational evidence of consensus clustering robustness.We provide motivation for further research on unsupervised learning for the CCPR problem. Early identification and detection of abnormal patterns is vital for a number of applications. In manufacturing for example, slide shifts and alterations of patterns might be indicative of some production process anomaly, such as machinery malfunction. Usually due to the continuous flow of data, monitoring of manufacturing processes and other types of applications requires automated control chart pattern recognition (CCPR) algorithms. Most of the CCPR literature consists of supervised classification algorithms. Fewer studies consider unsupervised versions of the problem. Despite the profound advantage of unsupervised methodology for less manual data labeling their use is limited due to the fact that their performance is not robust enough and might vary significantly from one algorithm to another. In this paper, we propose the use of a consensus clustering framework that takes care of this shortcoming and produces results that are robust with respect to the chosen pool of algorithms. Computational results show that the proposed method achieves not less than 79.10% G-mean with most of test instances achieving higher than 90%. This happens even when in the algorithmic pool are included algorithms with performance less than 15%. To our knowledge, this is the first paper proposing an unsupervised consensus learning approach in CCPR. The proposed approach is promising and provides a new research direction in unsupervised CCPR literature.

[1]  H. Brian Hwarng,et al.  Boltzmann machines that learn to recognize patterns on control charts , 1992 .

[2]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[3]  Fei Liu,et al.  Control chart pattern recognition using an integrated model based on binary-tree support vector machine , 2015 .

[4]  Santo Fortunato,et al.  Consensus clustering in complex networks , 2012, Scientific Reports.

[5]  Norma Faris Hubele,et al.  Back-propagation pattern recognizers for X¯ control charts: methodology and performance , 1993 .

[6]  Noriyoshi Sukegawa,et al.  Lagrangian relaxation and pegging test for the clique partitioning problem , 2013, Adv. Data Anal. Classif..

[7]  Stan Matwin,et al.  Machine Learning for the Detection of Oil Spills in Satellite Radar Images , 1998, Machine Learning.

[8]  Jianjun Shi,et al.  Automatic feature extraction of waveform signals for in-process diagnostic performance improvement , 2001, J. Intell. Manuf..

[9]  Lifeng Xi,et al.  A neural network ensemble-based model for on-line monitoring and diagnosis of out-of-control signals in multivariate manufacturing processes , 2009, Expert Syst. Appl..

[10]  Ratna Babu Chinnam,et al.  Robust kernel distance multivariate control chart using support vector principles , 2008 .

[11]  Yu Guo,et al.  Identification and quantification of concurrent control chart patterns using extreme-point symmetric mode decomposition and extreme learning machines , 2015, Neurocomputing.

[12]  Arthur Gretton,et al.  An online support vector machine for abnormal events detection , 2006, Signal Process..

[13]  Tian-Shyug Lee,et al.  A multi-stage control chart pattern recognition scheme based on independent component analysis and support vector machine , 2014, Journal of Intelligent Manufacturing.

[14]  Bin Wu,et al.  A neural network ensemble model for on-line monitoring of process mean and variance shifts in correlated processes , 2010, Expert Syst. Appl..

[15]  Way Kuo,et al.  Identification of control chart patterns using wavelet filtering and robust fuzzy clustering , 2007, J. Intell. Manuf..

[16]  H. B. Hwarng.  Multilayer perceptions for detecting cyclic data on control charts , 1995 .

[17]  Yi-Chih Hsieh,et al.  A neural network based model for abnormal pattern recognition of control charts , 1999 .

[18]  Marzuki Khalid,et al.  A Hybrid Artificial Neural Network-Naive Bayes for solving imbalanced dataset problems in semiconductor manufacturing test process , 2011, 2011 11th International Conference on Hybrid Intelligent Systems (HIS).

[19]  Chih-Chou Chiu,et al.  Using radial basis function neural networks to recognize shifts in correlated manufacturing process parameters , 1998 .

[20]  Gwilym M. Jenkins,et al.  Time series analysis, forecasting and control , 1971 .

[21]  Kai Yang,et al.  Application of artificial neural network to identify non-random variation patterns on the run chart in automotive assembly process , 2003 .

[22]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[23]  Ke Chen,et al.  On Coresets for k-Median and k-Means Clustering in Metric and Euclidean Spaces and Their Applications , 2009, SIAM J. Comput..

[24]  Seoung Bum Kim,et al.  Data mining model-based control charts for multivariate and autocorrelated processes , 2012, Expert Syst. Appl..

[25]  N F Hubele,et al.  X¯ CONTROL CHART PATTERN IDENTIFICATION THROUGH EFFICIENT OFF-LINE NEURAL NETWORK TRAINING , 1993 .

[26]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[27]  Valentina Eigner-Pitto,et al.  ChemProspector and generic structures: advanced mining and searching of chemical content , 2012, Journal of Cheminformatics.

[28]  Kuo-Ko Huang,et al.  Identifying Process Status Changes via Integration of Independent Component Analysis and Support Vector Machine , 2013 .

[29]  Petros Xanthopoulos,et al.  A Review on Consensus Clustering Methods , 2014 .

[30]  Khaled S. Al-Sultan,et al.  Computational experience on four algorithms for the hard clustering problem , 1996, Pattern Recognit. Lett..

[31]  Fionn Murtagh,et al.  A Survey of Recent Advances in Hierarchical Clustering Algorithms , 1983, Comput. J..

[32]  Stan Matwin,et al.  Learning When Negative Examples Abound , 1997, ECML.

[33]  Shokri Z. Selim,et al.  K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Charu C. Aggarwal,et al.  Outlier Detection for Temporal Data: A Survey , 2014, IEEE Transactions on Knowledge and Data Engineering.

[35]  T. Warren Liao,et al.  Clustering of time series data - a survey , 2005, Pattern Recognit..

[36]  Pedro Veiga,et al.  A retrospective view of statistical quality control research and identification of emerging trends: a bibliometric analysis , 2016 .

[37]  Sandro Vega-Pons,et al.  A Survey of Clustering Ensemble Algorithms , 2011, Int. J. Pattern Recognit. Artif. Intell..

[38]  W. A. Shewhart,et al.  Economic quality control of manufactured product , 1930 .

[39]  Atsuo Kawamura,et al.  Deadbeat control of PWM inverter with modified pulse patterns for uninterruptible power supply , 1988 .

[40]  Jionghua Jin,et al.  Automatic feature extraction of waveform signals for in-process diagnostic performance improvement , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[41]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[42]  Kamesh Munagala,et al.  Local Search Heuristics for k-Median and Facility Location Problems , 2004, SIAM J. Comput..

[43]  Ratna Babu Chinnam,et al.  Support vector machines for recognizing shifts in correlated and other manufacturing processes , 2002 .

[44]  Tawfik T. El-Midany,et al.  A proposed framework for control chart pattern recognition in multivariate process using artificial neural networks , 2010, Expert Syst. Appl..

[45]  A. Hoffman,et al.  Lower bounds for the partitioning of graphs , 1973 .

[46]  Marcus B. Perry,et al.  Control chart pattern recognition using back propagation artificial neural networks , 2001 .

[47]  Yoshiko Wakabayashi,et al.  A cutting plane algorithm for a clustering problem , 1989, Math. Program..

[48]  Alessia Saggese,et al.  Dynamic Scene Understanding for Behavior Analysis Based on String Kernels , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[50]  P. Young,et al.  Time series analysis, forecasting and control , 1972, IEEE Transactions on Automatic Control.

[51]  Jun Lv,et al.  Recognition of concurrent control chart patterns using wavelet transform decomposition and multiclass support vector machines , 2013, Comput. Ind. Eng..

[52]  Xenofontas A. Dimitropoulos,et al.  Histogram-based traffic anomaly detection , 2009, IEEE Transactions on Network and Service Management.

[53]  Naomie Salim,et al.  Voting-based consensus clustering for combining multiple clusterings of chemical structures , 2012, Journal of Cheminformatics.

[54]  Miin-Shen Yang,et al.  A control chart pattern recognition system using a statistical correlation coefficient method , 2005, Comput. Ind. Eng..

[55]  Amjed M. Al-Ghanim,et al.  An unsupervised learning neural algorithm for identifying process behavior on control charts and a comparison with supervised learning approaches , 1997 .

[56]  Amjed M. Al-Ghanim,et al.  Unnatural pattern recognition on control charts using correlation analysis techniques , 1995 .

[57]  F. Zorriassatine,et al.  Novelty detection for practical pattern recognition in condition monitoring of multivariate processes: a case study , 2005 .

[58]  Petros Xanthopoulos,et al.  A weighted support vector machine method for control chart pattern recognition , 2014, Comput. Ind. Eng..

[59]  Pierre Hansen,et al.  A VNS heuristic for escaping local extrema entrapment in normalized cut clustering , 2012, Pattern Recognit..

[60]  Ahmed Ghorbel,et al.  A survey of control-chart pattern-recognition literature (1991-2010) based on a new conceptual classification scheme , 2012, Comput. Ind. Eng..

[61]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .