A survey on feature selection methods for mixed data

Feature Selection for mixed data is an active research area with many applications in practical problems where numerical and non-numerical features describe the objects of study. This paper provides the first comprehensive and structured revision of the existing supervised and unsupervised feature selection methods for mixed data reported in the literature. Additionally, we present an analysis of the main characteristics, advantages, and disadvantages of the feature selection methods reviewed in this survey and discuss some important open challenges and potential future research opportunities in this field.

[1]  Luis E. Zárate,et al.  Categorical data clustering: What similarity measure to recommend? , 2015, Expert Syst. Appl..

[2]  Ulrike von Luxburg,et al.  A tutorial on spectral clustering , 2007, Stat. Comput..

[3]  Mohammad Ali Zare Chahooki,et al.  A Survey on semi-supervised feature selection methods , 2017, Pattern Recognit..

[4]  Verónica Bolón-Canedo,et al.  A review of feature selection methods in medical applications , 2019, Comput. Biol. Medicine.

[5]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Zhexue Huang,et al.  CLUSTERING LARGE DATA SETS WITH MIXED NUMERIC AND CATEGORICAL VALUES , 1997 .

[7]  Jaya Sil,et al.  Simultaneous feature selection and clustering with mixed features by multi objective genetic algorithm , 2014, Int. J. Hybrid Intell. Syst..

[8]  Monalisa Sarma,et al.  Two-stage approach to feature set optimization for unsupervised dataset with heterogeneous attributes , 2021, Expert Syst. Appl..

[9]  E. George,et al.  APPROACHES FOR BAYESIAN VARIABLE SELECTION , 1997 .

[10]  Qinghua Hu,et al.  Mixed feature selection based on granulation and approximation , 2008, Knowl. Based Syst..

[11]  S B Kotsiantis,et al.  RETRACTED ARTICLE: Feature selection for machine learning classification problems: a recent overview , 2014, Artificial Intelligence Review.

[12]  Daphne Koller,et al.  Toward Optimal Feature Selection , 1996, ICML.

[13]  Alexander J. Hartemink,et al.  Principled computational methods for the validation discovery of genetic regulatory networks , 2001 .

[14]  Matthieu Marbac,et al.  VarSelLCM: an R/C++ package for variable selection in model-based clustering of mixed-data with missing values , 2018, Bioinform..

[15]  Gautam Biswas,et al.  Unsupervised Learning with Mixed Numeric and Nominal Data , 2002, IEEE Trans. Knowl. Data Eng..

[16]  V. N. Sastry,et al.  Unsupervised feature ranking based on representation entropy , 2012, 2012 1st International Conference on Recent Advances in Information Technology (RAIT).

[17]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[18]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[19]  Qinghua Hu,et al.  Neighborhood rough set based heterogeneous feature subset selection , 2008, Inf. Sci..

[20]  Huan Liu,et al.  Efficient Feature Selection via Analysis of Relevance and Redundancy , 2004, J. Mach. Learn. Res..

[21]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[22]  Jiye Liang,et al.  An efficient feature selection algorithm for hybrid data , 2016, Neurocomputing.

[23]  Junghye Lee,et al.  Markov blanket-based universal feature selection for classification and regression of mixed-type data , 2020, Expert Syst. Appl..

[24]  Pramod Kumar Singh,et al.  A Survey on Filter Techniques for Feature Selection in Text Mining , 2012, SocProS.

[25]  S. S. Wilks.  The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses , 1938 .

[26]  E. Fowlkes,et al.  Variable selection in clustering , 1988 .

[27]  Bing Xue,et al.  A survey on feature selection approaches for clustering , 2020, Artificial Intelligence Review.

[28]  Huaici Zhao,et al.  A Discretization Algorithm of Continuous Attributes Based on Supervised Clustering , 2009, 2009 Chinese Conference on Pattern Recognition.

[29]  Sheng-yi Jiang,et al.  Efficient feature selection based on correlation measure between continuous and discrete features , 2016, Inf. Process. Lett..

[30]  Jesús Ariel Carrasco-Ochoa,et al.  A Supervised Filter Feature Selection Method for Mixed Data Based on the Spectral Gap Score , 2019, MCPR.

[31]  Tommi S. Jaakkola,et al.  CRAFT: ClusteR-specific Assorted Feature selecTion , 2015, AISTATS.

[32]  Michel Verleysen,et al.  The permutation test for feature selection by mutual information , 2006, ESANN.

[33]  Qiang Shen,et al.  Centre for Intelligent Systems and Their Applications Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Sets and Systems ( ) – Fuzzy–rough Attribute Reduction with Application to Web Categorization , 2022 .

[34]  E. George,et al.  Journal of the American Statistical Association is currently published by American Statistical Association. , 2007 .

[35]  Nanyan Liu.  The Research of Intrusion Detection Based on Mixed Clustering Algorithm , 2012, ISICA.

[36]  Jerzy W. Grzymala-Busse,et al.  Global discretization of continuous attributes as preprocessing for machine learning , 1996, Int. J. Approx. Reason..

[37]  Michel Verleysen,et al.  An Hybrid Approach to Feature Selection for Mixed Categorical and Continuous Data , 2011, KDIR.

[38]  Julie Ducreux,et al.  Feasibility of a molecular diagnosis of arthritis based on the iIdentification of specific transcriptomic profiles in knee synovial biopsies , 2011 .

[39]  Sen Liang,et al.  A Review of Matched-pairs Feature Selection Methods for Gene Expression Data Analysis , 2018, Computational and structural biotechnology journal.

[40]  Pierre Dupont,et al.  Kernel methods for heterogeneous feature selection , 2015, Neurocomputing.

[41]  Huan Liu,et al.  Feature Selection for Clustering: A Review , 2018, Data Clustering: Algorithms and Applications.

[42]  Dong Hyun Jeong,et al.  Designing a Feature Selection Technique for Analyzing Mixed Data , 2020, 2020 10th Annual Computing and Communication Workshop and Conference (CCWC).

[43]  Zdzis?aw Pawlak,et al.  Rough sets , 2005, International Journal of Computer & Information Sciences.

[44]  Joshua Zhexue Huang,et al.  Extensions to the k-Means Algorithm for Clustering Large Data Sets with Categorical Values , 1998, Data Mining and Knowledge Discovery.

[45]  Weihua Xu,et al.  Incremental approaches for heterogeneous feature selection in dynamic ordered data , 2020, Inf. Sci..

[46]  Manoranjan Dash,et al.  Dimensionality reduction of unsupervised data , 1997, Proceedings Ninth IEEE International Conference on Tools with Artificial Intelligence.

[47]  Chong-Ho Choi,et al.  Input Feature Selection by Mutual Information Based on Parzen Window , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Verónica Bolón-Canedo,et al.  Feature selection for high-dimensional data , 2016, Progress in Artificial Intelligence.

[49]  M. Punithavalli,et al.  Survey on Feature Selection in Document Clustering , 2011 .

[50]  Haitao Liu,et al.  A hybrid feature selection scheme for mixed attributes data , 2013 .

[51]  Degang Chen,et al.  Attribute Reduction for Heterogeneous Data Based on the Combination of Classical and Fuzzy Rough Set Models , 2014, IEEE Transactions on Fuzzy Systems.

[52]  Donald C. Wunsch,et al.  Clustering Data of Mixed Categorical and Numerical Type With Unsupervised Feature Learning , 2015, IEEE Access.

[53]  Hiroshi Motoda,et al.  Computational Methods of Feature Selection , 2022 .

[54]  Carla E. Brodley,et al.  Feature Selection for Unsupervised Learning , 2004, J. Mach. Learn. Res..

[55]  Habibollah Haron,et al.  Supervised, Unsupervised, and Semi-Supervised Feature Selection: A Review on Gene Selection , 2016, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[56]  Jian Weng,et al.  Feature selection for text classification: A review , 2018, Multimedia Tools and Applications.

[57]  Fang Liu,et al.  A Survey For Study of Feature Selection Based on Mutual Information , 2018, 2018 9th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[58]  Pablo A. Estévez,et al.  A review of feature selection methods based on mutual information , 2013, Neural Computing and Applications.

[59]  Sankar K. Pal,et al.  Pattern Recognition Algorithms for Data Mining , 2004 .

[60]  Charu C. Aggarwal,et al.  Data Clustering: Algorithms and Applications , 2014 .

[61]  A. Kraskov,et al.  Estimating mutual information. , 2003, Physical review. E, Statistical, nonlinear, and soft matter physics.

[62]  C. Hennig,et al.  How to find an appropriate clustering for mixed‐type variables with application to socio‐economic stratification , 2013 .

[63]  Jesús Ariel Carrasco-Ochoa,et al.  A new Unsupervised Spectral Feature Selection Method for mixed data: A filter approach , 2017, Pattern Recognit..

[64]  Alexander R. De Leon,et al.  Analysis of Mixed Data : Methods & Applications , 2013 .

[65]  Igor Kononenko,et al.  Estimating Attributes: Analysis and Extensions of RELIEF , 1994, ECML.

[66]  Huan Liu,et al.  Feature Selection for Clustering , 2000, Encyclopedia of Database Systems.

[67]  Francisco Herrera,et al.  A Survey of Discretization Techniques: Taxonomy and Empirical Analysis in Supervised Learning , 2013, IEEE Transactions on Knowledge and Data Engineering.

[68]  Nadia Essoussi,et al.  MapReduce-based k-prototypes clustering method for big data , 2015, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[69]  Verónica Bolón-Canedo,et al.  Feature Selection for High-Dimensional Data , 2015, Artificial Intelligence: Foundations, Theory, and Algorithms.

[70]  Jiye Liang,et al.  Determining the number of clusters using information entropy for mixed data , 2012, Pattern Recognit..

[71]  Huan Liu,et al.  Chi2: feature selection and discretization of numeric attributes , 1995, Proceedings of 7th IEEE International Conference on Tools with Artificial Intelligence.

[72]  Oksam Chae,et al.  Simultaneous feature selection and discretization based on mutual information , 2019, Pattern Recognit..

[73]  Kezhi Mao,et al.  Feature selection algorithm for mixed data with both nominal and continuous features , 2007, Pattern Recognit. Lett..

[74]  Xin Liu,et al.  Document clustering based on non-negative matrix factorization , 2003, SIGIR.

[75]  Yan Su,et al.  A Coupled User Clustering Algorithm Based on Mixed Data for Web-Based Learning Systems , 2015 .

[76]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[77]  Lukasz A. Kurgan,et al.  CAIM discretization algorithm , 2004, IEEE Transactions on Knowledge and Data Engineering.

[78]  Fátima Barceló-Rico,et al.  Geometrical codification for clustering mixed categorical and numerical databases , 2012, Journal of Intelligent Information Systems.

[79]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[80]  Charu C. Aggarwal,et al.  Data Clustering , 2013 .

[81]  Nikola Bogunovic,et al.  A review of feature selection methods with applications , 2015, 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[82]  J. Ruiz-Shulcloper,et al.  Pattern recognition with mixed and incomplete data , 2008, Pattern Recognition and Image Analysis.

[83]  Michel Verleysen,et al.  Mutual information based feature selection for mixed data , 2011, ESANN.

[84]  Huan Liu,et al.  Feature selection for classification: A review , 2014 .

[85]  Salvatore Greco,et al.  Rough sets theory for multicriteria decision analysis , 2001, Eur. J. Oper. Res..

[86]  Chi-Hyuck Jun,et al.  Rough set model based feature selection for mixed-type data with feature space decomposition , 2018, Expert Syst. Appl..

[87]  Michel Verleysen,et al.  A Mutual Information estimator for continuous and discrete variables applied to Feature Selection and Classification problems , 2016, Int. J. Comput. Intell. Syst..

[88]  Marianthi Markatou,et al.  Distance Metrics and Clustering Methods for Mixed‐type Data , 2018, International Statistical Review.

[89]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[90]  José Fco. Martínez-Trinidad,et al.  A Supervised Filter Feature Selection method for mixed data based on Spectral Feature Selection and Information-theory redundancy analysis , 2020, Pattern Recognit. Lett..

[91]  Hiroshi Motoda,et al.  Feature Selection for Knowledge Discovery and Data Mining , 1998, The Springer International Series in Engineering and Computer Science.

[92]  M. Marbac,et al.  Variable selection for mixed data clustering: a model-based approach , 2017, 1703.02293.

[93]  Yudong D. He,et al.  Gene expression profiling predicts clinical outcome of breast cancer , 2002, Nature.

[94]  Rajashree Dash,et al.  Comparative Analysis of Supervised and Unsupervised Discretization Techniques , 2011 .

[95]  Shulin Wang,et al.  Feature selection in machine learning: A new perspective , 2018, Neurocomputing.

[96]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[97]  Tommy W. S. Chow,et al.  Heterogeneous feature subset selection using mutual information-based feature transformation , 2015, Neurocomputing.

[98]  Qi Mao,et al.  Feature selection for unsupervised learning through local learning , 2015, Pattern Recognit. Lett..

[99]  Hugues Bersini,et al.  A Survey on Filter Techniques for Feature Selection in Gene Expression Microarray Analysis , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[100]  Jianyu Miao,et al.  A Survey on Feature Selection , 2016 .

[101]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[102]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[103]  José Fco. Martínez-Trinidad,et al.  A review of unsupervised feature selection methods , 2019, Artificial Intelligence Review.

[104]  Huan Liu,et al.  Redundancy based feature selection for microarray data , 2004, KDD.

[105]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[106]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[107]  R. Voigt,et al.  Clustering and variable selection in the presence of mixed variable types and missing data , 2017, Statistics in medicine.

[108]  Huan Liu,et al.  Spectral Feature Selection for Data Mining , 2011 .

[109]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[110]  M Cieplak.  蛋白質の折りたたみにおける協調性と接触秩序 | 文献情報 | J-GLOBAL 科学技術総合リンクセンター , 2004 .

[111]  Qinghua Hu,et al.  A Novel Algorithm for Finding Reducts With Fuzzy Rough Sets , 2012, IEEE Transactions on Fuzzy Systems.

[112]  Constantin F. Aliferis,et al.  Algorithms for Large Scale Markov Blanket Discovery , 2003, FLAIRS.

[113]  T. B. Murphy,et al.  Variable selection methods for model-based clustering , 2017, 1707.00306.

[114]  B. Chandra,et al.  An efficient statistical feature selection approach for classification of gene expression data , 2011, J. Biomed. Informatics.

[115]  Zhihao Wang,et al.  An Improved Kernel Clustering Algorithm for Mixed-Type Data in Network Forensic , 2016 .

[116]  Tao Li,et al.  Recent advances in feature selection and its applications , 2017, Knowledge and Information Systems.

[117]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[118]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[119]  Xuelong Li,et al.  Feature selection with multi-view data: A survey , 2019, Inf. Fusion.

[120]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[121]  Kezhi Mao,et al.  Feature Selection Algorithm for Data with Both Nominal and Continuous Features , 2005, PAKDD.

[122]  Andrew K. C. Wong,et al.  Synthesizing Statistical Knowledge from Incomplete Mixed-Mode Data , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[123]  Gregory F. Cooper,et al.  A latent variable model for multivariate discretization , 1999, AISTATS.

[124]  J. F. Chin,et al.  Feature selection in multimedia: The state-of-the-art review , 2017, Image Vis. Comput..

[125]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[126]  K. P. Singh,et al.  Support vector machines in water quality management. , 2011, Analytica chimica acta.

[127]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[128]  Huan Liu,et al.  Challenges of Feature Selection for Big Data Analytics , 2016, IEEE Intelligent Systems.

[129]  Shehroz S. Khan,et al.  Survey of State-of-the-Art Mixed Data Clustering Algorithms , 2018, IEEE Access.

[130]  Xiao Zhang,et al.  Feature selection in mixed data: A method using a novel fuzzy rough set-based information entropy , 2016, Pattern Recognit..