Fast unsupervised feature selection based on the improved binary ant system and mutation strategy

The “curse of dimensionality” issue caused by high-dimensional datasets not only imposes high memory and computational costs but also deteriorates the capability of learning methods. The main purpose of feature selection is to reduce the dimensionality of these datasets by discarding redundant and irrelevant features, which improves the performance of the learning algorithm. In this paper, a new feature selection algorithm, referred to as FSBACOM, was presented based on the binary ant system (BAS). The proposed method sought to improve feature selection by decreasing redundancy and achieved an optimum solution by increasing search space in a short time. For this purpose, the features were organized sequentially in a circular graph, where each feature was connected to the next one with two select/deselect edges. The proposed representation of the search space reduced computational time significantly, particularly on the high-dimensional datasets. Inspired from genetic algorithm and simulated annealing, a damped mutation strategy was introduced to avoid falling into local optima. In addition, a new idea was utilized to reduce the redundancy between selected features as far as possible. The performance of the proposed algorithm was compared to that of state-of-the-art feature selection algorithms using different classifiers on real-world datasets. The experimental results confirmed that FSBACOM significantly reduces computational time and achieves better performance than other feature selection methods.

[1]  Zenglin Xu,et al.  Discriminative Semi-Supervised Feature Selection Via Manifold Regularization , 2009, IEEE Transactions on Neural Networks.

[2]  Li-Yeh Chuang,et al.  Chaotic maps based on binary particle swarm optimization for feature selection , 2011, Appl. Soft Comput..

[3]  Yumin Chen,et al.  A rough set approach to feature selection based on power set tree , 2011, Knowl. Based Syst..

[4]  Yi Jiang,et al.  Eigenvalue Sensitive Feature Selection , 2011, ICML.

[5]  Filiberto Pla,et al.  Supervised feature selection by clustering using conditional mutual information-based distances , 2010, Pattern Recognit..

[6]  Li-Yeh Chuang,et al.  Improved binary particle swarm optimization using catfish effect for feature selection , 2011, Expert Syst. Appl..

[7]  Yixin Chen,et al.  Efficient ant colony optimization for image feature selection , 2013, Signal Process..

[8]  Pedro Larrañaga,et al.  A review of feature selection techniques in bioinformatics , 2007, Bioinform..

[9]  J. Mesirov,et al.  Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[10]  Francis R. Bach,et al.  Structured Variable Selection with Sparsity-Inducing Norms , 2009, J. Mach. Learn. Res..

[11]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[12]  Leslie S. Smith,et al.  Feature subset selection in large dimensionality domains , 2010, Pattern Recognit..

[13]  Hossein Nezamabadi-pour,et al.  An advanced ACO algorithm for feature subset selection , 2015, Neurocomputing.

[14]  Sameer A. Nene,et al.  Columbia Object Image Library (COIL100) , 1996 .

[15]  Huan Liu,et al.  Spectral feature selection for supervised and unsupervised learning , 2007, ICML '07.

[16]  Jacob Benesty,et al.  Noise Reduction in Speech Processing , 2009 .

[17]  Mário A. T. Figueiredo,et al.  Efficient feature selection filters for high-dimensional data , 2012, Pattern Recognit. Lett..

[18]  Nikola Bogunovic,et al.  A review of feature selection methods with applications , 2015, 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[19]  Yongdai Kim,et al.  Gradient LASSO for feature selection , 2004, ICML.

[20]  Chidchanok Lursinsap,et al.  A Discrimination Analysis for Unsupervised Feature Selection via Optic Diffraction Principle , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[21]  T. Warren Liao,et al.  Medical data mining by fuzzy modeling with selected features , 2008, Artif. Intell. Medicine.

[22]  Marcel J. T. Reinders,et al.  Random subspace method for multivariate feature selection , 2006, Pattern Recognit. Lett..

[23]  Lei Wang,et al.  On Similarity Preserving Feature Selection , 2013, IEEE Transactions on Knowledge and Data Engineering.

[24]  Bart Baesens,et al.  Editorial survey: swarm intelligence for data mining , 2010, Machine Learning.

[25]  Parham Moradi,et al.  Relevance-redundancy feature selection based on ant colony optimization , 2015, Pattern Recognit..

[26]  Feiping Nie,et al.  Discriminative Least Squares Regression for Multiclass Classification and Feature Selection , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[27]  Jacob Benesty,et al.  Pearson Correlation Coefficient , 2009 .

[28]  Min Kong,et al.  Introducing a Binary Ant Colony Optimization , 2006, ANTS Workshop.

[29]  Ouahab Kadri,et al.  Fault diagnosis of rotary kiln using SVM and binary ACO , 2012 .

[30]  Jidong Zhao,et al.  Locality sensitive semi-supervised feature selection , 2008, Neurocomputing.

[31]  Anrong Yang,et al.  Feature selection using data envelopment analysis , 2014, Knowl. Based Syst..

[32]  Deng Cai,et al.  Laplacian Score for Feature Selection , 2005, NIPS.

[33]  Selma Ayse Özel,et al.  A hybrid approach of differential evolution and artificial bee colony for feature selection , 2016, Expert Syst. Appl..

[34]  Luca Maria Gambardella,et al.  Ant colony system: a cooperative learning approach to the traveling salesman problem , 1997, IEEE Trans. Evol. Comput..

[35]  Thomas G. Dietterich,et al.  Efficient Algorithms for Identifying Relevant Features , 1992 .

[36]  Kazuyuki Murase,et al.  A new local search based hybrid genetic algorithm for feature selection , 2011, Neurocomputing.

[37]  Jian Zhang,et al.  Semi-supervised feature selection based on local discriminative information , 2016, Neurocomputing.

[38]  Ruisheng Zhang,et al.  A BPSO-SVM algorithm based on memory renewal and enhanced mutation mechanisms for feature selection , 2017, Appl. Soft Comput..

[39]  Zi Huang,et al.  Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence ℓ2,1-Norm Regularized Discriminative Feature Selection for Unsupervised Learning , 2022 .

[40]  Duoqian Miao,et al.  A rough set approach to feature selection based on ant colony optimization , 2010, Pattern Recognit. Lett..

[41]  Behrouz Minaei-Bidgoli,et al.  Optimizing multi-objective PSO based feature selection method using a feature elitism mechanism , 2018, Expert Syst. Appl..

[42]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[43]  Jian-Bo Yang,et al.  Feature Selection Using Probabilistic Prediction of Support Vector Regression , 2011, IEEE Transactions on Neural Networks.

[44]  Gang Qu,et al.  Modified genetic algorithm-based feature selection combined with pre-trained deep neural network for demand forecasting in outpatient department , 2017, Expert Syst. Appl..

[45]  Parham Moradi,et al.  An unsupervised feature selection algorithm based on ant colony optimization , 2014, Eng. Appl. Artif. Intell..

[46]  Fuhui Long,et al.  Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy , 2003, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Mohammad-Reza Feizi-Derakhshi,et al.  Feature selection using Forest Optimization Algorithm , 2016, Pattern Recognit..

[48]  Parham Moradi,et al.  Gene selection for microarray data classification using a novel ant colony optimization , 2015, Neurocomputing.

[49]  Ratna Babu Chinnam,et al.  mr2PSO: A maximum relevance minimum redundancy feature selection method based on swarm intelligence for support vector machine classification , 2011, Inf. Sci..

[50]  Byung Ro Moon,et al.  Hybrid Genetic Algorithms for Feature Selection , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  M. Friedman A Comparison of Alternative Tests of Significance for the Problem of $m$ Rankings , 1940 .

[52]  Min Kong,et al.  A Binary Ant Colony Optimization for the Unconstrained Function Optimization Problem , 2005, CIS.

[53]  Amit Kumar Das,et al.  A new hybrid feature selection approach using feature association map for supervised and unsupervised classification , 2017, Expert Syst. Appl..

[54]  Cheng-Lung Huang,et al.  Handling sequential pattern decay: Developing a two-stage collaborative recommender system , 2009, Electron. Commer. Res. Appl..

[55]  Zhiwei Ye,et al.  A feature selection method based on modified binary coded ant colony optimization algorithm , 2016, Appl. Soft Comput..

[56]  Mário A. T. Figueiredo,et al.  An unsupervised approach to feature discretization and selection , 2012, Pattern Recognit..

[57]  U. Alon,et al.  Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[58]  Jin-Ho Kim,et al.  A Novel Binary Ant Colony Optimization: Application to the Unit Commitment Problem of Power Systems , 2011 .

[59]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[60]  Constantine Kotropoulos,et al.  Feature Selection Based on Mutual Correlation , 2006, CIARP.