Multi-Criteria Ant Feature Selection Using Fuzzy Classifiers

One of the most important techniques in data preprocessing for data mining is feature selection. Real-world data analysis, data mining, classification and modeling problems usually involve a large number of candidate inputs or features. Less relevant or highly correlated features decrease, in general, the classification accuracy, and enlarge the complexity of the classifier. The goal is to find a reduced set of features that reveals the best classification accuracy for a fuzzy classifier. This chapter presents an ant colony optimization (ACO) algorithm for feature selection, which minimizes two objectives: the number of features and the error classification. Two pheromone matrices and two different heuristics are used for each objective. The performance of the method is compared to other feature selection methods, revealing similar or higher performance.

[1]  Thomas Stützle,et al.  The Ant Colony Optimization Metaheuristic: Algorithms, Applications, and Advances , 2003 .

[2]  Robert John,et al.  Developments in soft computing , 2001 .

[3]  Witold Pedrycz,et al.  Foundations of Fuzzy Logic and Soft Computing, 12th International Fuzzy Systems Association World Congress, IFSA 2007, Cancun, Mexico, June 18-21, 2007, Proceedings , 2007, IFSA.

[4]  Günther R. Raidl,et al.  Letting ants labeling point features [sic.: for 'labeling' read 'label'] , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[5]  János Abonyi,et al.  Learning Fuzzy Classification Rules from Data , 2001 .

[6]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[7]  János Abonyi,et al.  Learning fuzzy classification rules from labeled data , 2003, Inf. Sci..

[8]  David G. Stork,et al.  Pattern Classification , 1973 .

[9]  Michael Schreyer.  Letting Ants Labeling Point Features , 2002 .

[10]  George J. Klir,et al.  Fuzzy sets and fuzzy logic - theory and applications , 1995 .

[11]  Yingxu Wang,et al.  On Cognitive Informatics , 2002, Proceedings First IEEE International Conference on Cognitive Informatics.

[12]  João Miguel da Costa Sousa,et al.  Decision tree search methods in fuzzy modeling and classification , 2007, Int. J. Approx. Reason..

[13]  Thomas A. Runkler,et al.  Ant Colony Optimization Applied to Feature Selection in Fuzzy Classifiers , 2007, IFSA.

[14]  Yu Wu,et al.  Theoretical study on attribute reduction of rough set theory: comparison of algebra and information views , 2004 .

[15]  Ferenc Szeifert,et al.  Supervised fuzzy clustering for the identification of fuzzy classifiers , 2003, Pattern Recognit. Lett..

[16]  Tsau Young Lin,et al.  Rough Set Methods and Applications , 2000 .

[17]  F. Glover,et al.  Handbook of Metaheuristics , 2019, International Series in Operations Research & Management Science.

[18]  Marco Dorigo,et al.  Optimization, Learning and Natural Algorithms , 1992 .

[19]  Yuchang Lu,et al.  Feature ranking in rough sets , 2003, AI Commun..

[20]  Magne Setnes,et al.  GA-fuzzy modeling and classification: complexity and performance , 2000, IEEE Trans. Fuzzy Syst..

[21]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[22]  Huan Liu,et al.  Toward integrating feature selection algorithms for classification and clustering , 2005, IEEE Transactions on Knowledge and Data Engineering.

[23]  Jan G. Bazan,et al.  Rough set algorithms in classification problem , 2000 .

[24]  Thomas A. Runkler,et al.  Fuzzy classification in ant feature selection , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[25]  Thomas Stützle,et al.  Ant Colony Optimization , 2009, EMO.

[26]  Qiang Shen,et al.  Fuzzy-rough data reduction with ant colony optimization , 2005, Fuzzy Sets Syst..

[27]  Thomas Stützle,et al.  Ant colony optimization: artificial ants as a computational intelligence technique , 2006 .

[28]  Olcay Boz,et al.  Feature Subset Selection by Using Sorted Feature Relevance , 2002, ICMLA.

[29]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[30]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[31]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[32]  Uzay Kaymak,et al.  Fuzzy Decision Making in Modeling and Control , 2002, World Scientific Series in Robotics and Intelligent Systems.

[33]  Thomas A. Runkler,et al.  Distributed optimisation of a logistic system and its suppliers using ant colonies , 2006, Int. J. Syst. Sci..

[34]  Michio Sugeno,et al.  A fuzzy-logic-based approach to qualitative modeling , 1993, IEEE Trans. Fuzzy Syst..

[35]  Jing-Yu Yang,et al.  Optimal discriminant plane for a small number of samples and design method of classifier on the plane , 1991, Pattern Recognit..

[36]  Huan Liu,et al.  Feature selection for clustering - a filter solution , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[37]  Christopher M. Bishop,et al.  A New Framework for Machine Learning , 2008, WCCI.

[38]  Sreeram Ramakrishnan,et al.  A hybrid approach for feature subset selection using neural networks and ant colony optimization , 2007, Expert Syst. Appl..

[39]  Thomas A. Runkler,et al.  Rescheduling and optimization of logistic processes using GA and ACO , 2008, Eng. Appl. Artif. Intell..

[40]  Qiang Shen,et al.  Centre for Intelligent Systems and Their Applications Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Rough Attribute Reduction with Application to Web Categorization Fuzzy Sets and Systems ( ) – Fuzzy–rough Attribute Reduction with Application to Web Categorization , 2022 .

[41]  Ahmed Al-Ani,et al.  Feature Subset Selection Using Ant Colony Optimization , 2008 .

[42]  M. Pazzani,et al.  ID2-of-3: Constructive Induction of M-of-N Concepts for Discriminators in Decision Trees , 1991 .

[43]  Xiangyang Wang,et al.  Feature selection based on rough sets and particle swarm optimization , 2007, Pattern Recognit. Lett..

[44]  C. A. Murthy,et al.  Unsupervised Feature Selection Using Feature Similarity , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  A. Atkinson.  Subset Selection in Regression , 1992 .

[46]  Sandip Sen,et al.  Using real-valued genetic algorithms to evolve rule sets for classification , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.