Parsimonious Modeling for Binary Classification of Quality in a High Conformance Manufacturing Environment

The world of big data is changing dramatically; in the domain of data mining, machine learning and pattern recognition, the feature access has grown from tens to hundreds or even thousands. This trend presents enormous challenges, specially for classification problems. In manufacturing, classification of quality is one of the most important applications; however, feature explosion, combined with high conformance production rates are two of the most important challenges for big data initiatives. Empirical evidence shows that discarding irrelevant or redundant features improves prediction, helps in understanding the system, reduces running time requirements, and reduces the effect of dimensionality. In this paper, the Hybrid Correlationand Ranking-based (HCR) and ReliefF filter feature elimination algorithms are presented as a wrapper method, which uses the Naive Bayes as the learning algorithm. To boost parsimony, the algorithms are combined with the Penalized Maximum Probability of Correct Decision – a model selection criterion – to develop a Hybrid Feature Selection and Pattern Recognition framework aimed at rare quality event detection. A flexible approach that can be widely applied to various machine learning algorithms.

[1]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[2]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[3]  Tae-Hyung Kim,et al.  Feature selection for manufacturing process monitoring using cross-validation , 2013 .

[4]  Andrew Y. Ng,et al.  On Feature Selection: Learning with Exponentially Many Irrelevant Features as Training Examples , 1998, ICML.

[5]  Rubén Morales-Menéndez,et al.  Machine Learning and Pattern Recognition Techniques for Information Extraction to Improve Production Control and Design Decisions , 2017, ICDM.

[6]  Bart De Moor,et al.  Hyperparameter Search in Machine Learning , 2015, ArXiv.

[7]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[8]  Ruben Morales-Menendez,et al.  Process-Monitoring-for-Quality—Applications , 2018 .

[9]  Azuraliza Abu Bakar,et al.  Naïve bayes variants in classification learning , 2010, 2010 International Conference on Information Retrieval & Knowledge Management (CAMP).

[10]  Ruben Morales-Menendez,et al.  Process-Monitoring-for-Quality — A Model Selection Criterion , 2018 .

[11]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[12]  Mark A. Hall,et al.  Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning , 1999, ICML.

[13]  Sylvain Arlot,et al.  A survey of cross-validation procedures for model selection , 2009, 0907.4728.

[14]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[15]  Carlos A. Escobar,et al.  Big Data-Driven Manufacturing—Process-Monitoring-for-Quality Philosophy , 2017 .

[16]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[17]  George C. Runger,et al.  Feature selection via regularized trees , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[18]  Inci Batmaz,et al.  A review of data mining applications for quality improvement in manufacturing industry , 2011, Expert Syst. Appl..

[19]  Ruben Morales-Menendez,et al.  Machine learning techniques for quality control in high conformance manufacturing environment , 2018 .

[20]  Richard Demo Souza,et al.  A Survey of Machine Learning Techniques Applied to Self-Organizing Cellular Networks , 2017, IEEE Communications Surveys & Tutorials.

[21]  Tom Fawcett,et al.  An introduction to ROC analysis , 2006, Pattern Recognit. Lett..

[22]  Max Kuhn,et al.  Applied Predictive Modeling , 2013 .

[23]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[24]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[25]  Eric R. Ziegel,et al.  Probability and Statistics for Engineering and the Sciences , 2004, Technometrics.

[26]  Fei Wang,et al.  Feature selection using feature ranking, correlation analysis and chaotic binary particle swarm optimization , 2014, 2014 IEEE 5th International Conference on Software Engineering and Service Science.

[27]  Wei Wang,et al.  Application of Global Optimization Methods for Feature Selection and Machine Learning , 2013 .

[28]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[29]  Paul S. Bradley,et al.  Feature Selection via Concave Minimization and Support Vector Machines , 1998, ICML.

[30]  Yoshua Bengio,et al.  Random Search for Hyper-Parameter Optimization , 2012, J. Mach. Learn. Res..

[31]  David R. Anderson,et al.  Model selection and multimodel inference : a practical information-theoretic approach , 2003 .