Rough cognitive ensembles

Abstract Rough Cognitive Networks are granular classifiers stemming from the hybridization of Fuzzy Cognitive Maps and Rough Set Theory. Such cognitive neural networks attempt to quantify the impact of rough granular constructs (i.e., the positive, negative and boundary regions of a target concept) over each decision class for the problem at hand. In rough classifiers, determining the precise granularity level is crucial to compute high prediction rates. Regrettably, learning the similarity threshold parameter requires reconstructing the information granules, which may be time-consuming. In this paper, we put forth a new multiclassifier system classifier named Rough Cognitive Ensembles. The proposed ensemble employs a collection of Rough Cognitive Networks as base classifiers, each operating at a different granularity level. This allows suppressing the requirement of learning a similarity threshold. We evaluate the granular ensemble with 140 traditional classification datasets using different heterogeneous distance functions. After comparing the proposed model to 15 well-known classifiers, the experimental evidence confirms that our scheme yields very promising classification rates.

[1]  Haijia Shi Best-first Decision Tree Learning , 2007 .

[2]  Jun Gao,et al.  A survey of neural network ensembles , 2005, 2005 International Conference on Neural Networks and Brain.

[3]  Witold Pedrycz,et al.  From Fuzzy Cognitive Maps to Granular Cognitive Maps , 2014, IEEE Trans. Fuzzy Syst..

[4]  Lech Polkowski,et al.  Granular Computing in Decision Approximation - An Application of Rough Mereology , 2015, Intelligent Systems Reference Library.

[5]  Yiyu Yao,et al.  Three-way decisions with probabilistic rough sets , 2010, Inf. Sci..

[6]  Jose L. Salmeron,et al.  Benchmarking main activation functions in fuzzy cognitive maps , 2009, Expert Syst. Appl..

[7]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[8]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[9]  Francis K. H. Quek,et al.  Attribute bagging: improving accuracy of classifier ensembles by using random feature subsets , 2003, Pattern Recognit..

[10]  Koen Vanhoof,et al.  Partitive granular Cognitive Maps to graded multilabel classification , 2016, 2016 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[11]  Bart Kosko,et al.  Hidden patterns in combined and adaptive knowledge networks , 1988, Int. J. Approx. Reason..

[12]  Salvatore J. Stolfo,et al.  The application of AdaBoost for distributed, scalable and on-line learning , 1999, KDD '99.

[13]  Witold Pedrycz,et al.  The design of granular classifiers: A study in the synergy of interval calculus and fuzzy sets in pattern recognition , 2008, Pattern Recognit..

[14]  Yiyu Yao,et al.  Three-Way Decision: An Interpretation of Rules in Rough Set Theory , 2009, RSKT.

[15]  M. Verleysen,et al.  Classification in the Presence of Label Noise: A Survey , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[17]  David G. Stork,et al.  Pattern Classification , 1973 .

[18]  Witold Pedrycz,et al.  The design of cognitive maps: A study in synergy of granular computing and evolutionary optimization , 2010, Expert Syst. Appl..

[19]  Witold Pedrycz,et al.  Description and classification of granular time series , 2015, Soft Comput..

[20]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[21]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[22]  Andrew K. C. Wong,et al.  Classification of Imbalanced Data: a Review , 2009, Int. J. Pattern Recognit. Artif. Intell..

[23]  Ajith Abraham,et al.  Rough Set Theory: A True Landmark in Data Analysis , 2009 .

[24]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[25]  Hecht-Nielsen Theory of the backpropagation neural network , 1989 .

[26]  Witold Pedrycz,et al.  Granular Computing: At the Junction of Rough Sets and Fuzzy Sets , 2008 .

[27]  Witold Pedrycz,et al.  Design of Fuzzy Cognitive Maps for Modeling Time Series , 2016, IEEE Transactions on Fuzzy Systems.

[28]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[29]  Witold Pedrycz,et al.  Granular computing: an introduction , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[30]  Thomas G. Dietterich Multiple Classifier Systems , 2000, Lecture Notes in Computer Science.

[31]  Ponnuthurai N. Suganthan,et al.  Ensemble Classification and Regression-Recent Developments, Applications and Future Directions [Review Article] , 2016, IEEE Computational Intelligence Magazine.

[32]  Eyke Hüllermeier,et al.  Graded Multilabel Classification: The Ordinal Case , 2010, ICML.

[33]  Hisao Ishibuchi,et al.  Voting in fuzzy rule-based systems for pattern classification problems , 1999, Fuzzy Sets Syst..

[34]  Jerzy W. Grzymala-Busse,et al.  Rough Sets , 1995, Commun. ACM.

[35]  Witold Pedrycz,et al.  From data to granular data and granular classifiers , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[36]  Dominik Ślęzak,et al.  Building Granular Systems - from Concepts to Applications , 2015, RSFDGrC.

[37]  Eibe Frank,et al.  Speeding Up Logistic Model Tree Induction , 2005, PKDD.

[38]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[39]  Bart Kosko,et al.  Fuzzy Cognitive Maps , 1986, Int. J. Man Mach. Stud..

[40]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[41]  Nikunj C. Oza,et al.  Decimated input ensembles for improved generalization , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[42]  Yiyu Yao,et al.  The superiority of three-way decisions in probabilistic rough set models , 2011, Inf. Sci..

[43]  Koen Vanhoof,et al.  Rough Cognitive Networks , 2016, Knowl. Based Syst..

[44]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[45]  Harry Zhang,et al.  A Fast Decision Tree Learning Algorithm , 2006, AAAI.

[46]  Albert Y. Zomaya,et al.  A Review of Ensemble Methods in Bioinformatics , 2010, Current Bioinformatics.

[47]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[48]  Koen Vanhoof,et al.  Hybrid Model Based on Rough Sets Theory and Fuzzy Cognitive Maps for Decision-Making , 2014, RSEISP.

[49]  Ron Kohavi,et al.  The Power of Decision Tables , 1995, ECML.

[50]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[51]  Koen Vanhoof,et al.  A Granular Intrusion Detection System Using Rough Cognitive Networks , 2016, Recent Advances in Computational Intelligence in Defense and Security.

[52]  Francesca Mangili,et al.  Should We Really Use Post-Hoc Tests Based on Mean-Ranks? , 2015, J. Mach. Learn. Res..

[53]  Eibe Frank,et al.  Logistic Model Trees , 2003, Machine Learning.

[54]  Andrzej Skowron,et al.  Rough-Neural Computing: Techniques for Computing with Words , 2004, Cognitive Technologies.

[55]  John G. Cleary,et al.  K*: An Instance-based Learner Using and Entropic Distance Measure , 1995, ICML.

[56]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[57]  M. Friedman The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[58]  Yali Amit,et al.  Shape Quantization and Recognition with Randomized Trees , 1997, Neural Computation.

[59]  Terry Ngo,et al.  Data mining: practical machine learning tools and technique, third edition by Ian H. Witten, Eibe Frank, Mark A. Hell , 2011, SOEN.

[60]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[61]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[62]  Marti A. Hearst Trends & Controversies: Support Vector Machines , 1998, IEEE Intell. Syst..

[63]  S. Sathiya Keerthi,et al.  Improvements to Platt's SMO Algorithm for SVM Classifier Design , 2001, Neural Computation.

[64]  Sotiris B. Kotsiantis,et al.  Decision trees: a recent overview , 2011, Artificial Intelligence Review.

[65]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[66]  Andrew W. Moore,et al.  Locally Weighted Learning for Control , 1997, Artificial Intelligence Review.

[67]  Witold Pedrycz,et al.  Granular Cognitive Maps reconstruction , 2014, 2014 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE).

[68]  Bogdan Gabrys,et al.  Metalearning: a survey of trends and technologies , 2013, Artificial Intelligence Review.

[69]  Francisco Herrera,et al.  An insight into classification with imbalanced data: Empirical results and current trends on using data intrinsic characteristics , 2013, Inf. Sci..

[70]  Nicu Sebe,et al.  Semisupervised learning of classifiers: theory, algorithms, and their application to human-computer interaction , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[71]  Witold Pedrycz,et al.  Granular Computing: Analysis and Design of Intelligent Systems , 2013 .

[72]  Lech Polkowski,et al.  On Granular Rough Computing: Factoring Classifiers Through Granulated Decision Systems , 2007, RSEISP.

[73]  Francisco Herrera,et al.  Self-labeled techniques for semi-supervised learning: taxonomy, software and empirical study , 2015, Knowledge and Information Systems.

[74]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.