Optimal Feature Subset Determination for High-Dimensional Datasets in Manufacturing Processes

Selecting the most relevant feature subset with good predictive performances and computational speed for high-dimensional datasets is usually challenging in manufacturing processes for product quality control. In this paper, an optimal feature subset determination approach for high-dimensional datasets is proposed. The algorithm starts with ranking the importance of individual features using the normalized mutual information concepts. Then, through optimizing a new evaluation metric, an initial relevant feature subset is obtained. Finally, redundant features are eliminated to help obtain the final selection results. The proposed method was successfully used in solving a real-world high-dimensional feature selection problem in the semiconductor industry. Comparisons were made with four other representative feature selection algorithms, including Relief-F, mRMR, FCBF, and IWFS in processing a number of datasets from different applications. It is demonstrated that the proposed method can automatically determine an optimal feature subset to achieve good average predictive accuracy with less computational resources.

[1]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.

[2]  Jieping Ye,et al.  Feature grouping and selection over an undirected graph , 2012, KDD.

[3]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[4]  Colas Schretter,et al.  Information-Theoretic Feature Selection in Microarray Data Using Variable Complementarity , 2008, IEEE Journal of Selected Topics in Signal Processing.

[5]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[7]  Rui Zhang,et al.  A novel feature selection method considering feature interaction , 2015, Pattern Recognit..

[8]  M. Yuan,et al.  Model selection and estimation in regression with grouped variables , 2006 .

[9]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[10]  Lluís A. Belanche Muñoz,et al.  Feature selection algorithms: a survey and experimental evaluation , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[11]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[12]  Huan Liu,et al.  Consistency-based search in feature selection , 2003, Artif. Intell..

[13]  David D. Lewis,et al.  Feature Selection and Feature Extraction for Text Categorization , 1992, HLT.

[14]  Belén Melián-Batista,et al.  High-dimensional feature selection via feature grouping: A Variable Neighborhood Search approach , 2016, Inf. Sci..

[15]  Jieping Ye,et al.  Moreau-Yosida Regularization for Grouped Tree Structure Learning , 2010, NIPS.

[16]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[17]  Larry A. Rendell,et al.  The Feature Selection Problem: Traditional Methods and a New Algorithm , 1992, AAAI.

[18]  Kewei Cheng,et al.  Feature Selection , 2016, ACM Comput. Surv..

[19]  F. Fleuret Fast Binary Feature Selection with Conditional Mutual Information , 2004, J. Mach. Learn. Res..

[20]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[21]  Xin Yao,et al.  A Survey on Evolutionary Computation Approaches to Feature Selection , 2016, IEEE Transactions on Evolutionary Computation.

[22]  Huan Liu,et al.  Feature selection for classification: A review , 2014 .

[23]  Daphne Koller,et al.  Toward Optimal Feature Selection , 1996, ICML.