A Novel Meta Learning Framework for Feature Selection using Data Synthesis and Fuzzy Similarity

This paper presents a novel meta learning framework for feature selection (FS) based on fuzzy similarity. The proposed method aims to recommend the best FS method from four candidate FS methods for any given dataset. This is achieved by firstly constructing a large training data repository using data synthesis. Six meta features that represent the characteristics of the training dataset are then extracted. The best FS method for each of the training datasets is used as the meta label. Both the meta features and the corresponding meta labels are subsequently used to train a classification model using a fuzzy similarity measure based framework. Finally the trained model is used to recommend the most suitable FS method for a given unseen dataset. This proposed method was evaluated based on eight public datasets of real-world applications. It successfully recommended the best method for five datasets and the second best method for one dataset, which outperformed any of the four individual FS methods. Besides, the proposed method is computationally efficient for algorithm selection, leading to negligible additional time for the feature selection process. Thus, the paper contributes a novel method for effectively recommending which feature selection method to use for any new given dataset.

[1]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[2]  Bogdan Gabrys,et al.  Metalearning: a survey of trends and technologies , 2013, Artificial Intelligence Review.

[3]  Andreas Dengel,et al.  Automatic classifier selection for non-experts , 2012, Pattern Analysis and Applications.

[4]  Joel Grus,et al.  Data Science from Scratch: First Principles with Python , 2015 .

[5]  Edward R. Dougherty,et al.  Performance of feature-selection methods in the classification of high-dimension data , 2009, Pattern Recognit..

[6]  Isabelle Guyon,et al.  Design of experiments for the NIPS 2003 variable selection benchmark , 2003 .

[7]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[8]  Ricardo Vilalta,et al.  Metalearning - Applications to Data Mining , 2008, Cognitive Technologies.

[9]  Verónica Bolón-Canedo,et al.  A review of feature selection methods on synthetic data , 2013, Knowledge and Information Systems.

[10]  Sven F. Crone,et al.  Meta-learning with neural networks and landmarking for forecasting model selection an empirical evaluation of different feature sets applied to industry data , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[11]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[12]  Jonathan M. Garibaldi,et al.  Performance Optimization of a Fuzzy Entropy Based Feature Selection and Classification Framework , 2018, 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[13]  Andreas Dengel,et al.  Meta-learning for evolutionary parameter optimization of classifiers , 2012, Machine Learning.

[14]  Timothy A. Gonsalves,et al.  Feature Selection for Text Classification Based on Gini Coefficient of Inequality , 2010, FSDM.

[15]  Yves Le Traon,et al.  Meta-Modelling Meta-Learning , 2019, 2019 ACM/IEEE 22nd International Conference on Model Driven Engineering Languages and Systems (MODELS).

[16]  Pasi Luukka,et al.  Similarity classifier with generalized mean applied to medical data , 2006, Comput. Biol. Medicine.

[17]  Robert LIN,et al.  NOTE ON FUZZY SETS , 2014 .

[18]  Melanie Hilario,et al.  Feature Selection for Meta-learning , 2001, PAKDD.

[19]  Jiawei Han,et al.  Data Mining: Concepts and Techniques , 2000 .

[20]  Andrey Filchenkov,et al.  Datasets meta-feature description for recommending feature selection algorithm , 2015, 2015 Artificial Intelligence and Natural Language and Information Extraction, Social Media and Web Search FRUCT Conference (AINL-ISMW FRUCT).

[21]  Huei Diana Lee,et al.  Metalearning for choosing feature selection algorithms in data mining: Proposal of a new framework , 2017, Expert Syst. Appl..

[22]  Pasi Luukka,et al.  A classifier based on the maximal fuzzy similarity in the generalized Lukasiewicz-structure , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).

[23]  Jonathan M. Garibaldi,et al.  A Novel Weighted Combination Method for Feature Selection using Fuzzy Sets , 2019, 2019 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[24]  Bogdan Gabrys,et al.  Meta-learning for time series forecasting and forecast combination , 2010, Neurocomputing.

[25]  Marco Cristani,et al.  Infinite Feature Selection , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Roberto Battiti,et al.  Using mutual information for selecting features in supervised neural net learning , 1994, IEEE Trans. Neural Networks.