Fuzzy-rough nearest neighbour classification and prediction

Nearest neighbour (NN) approaches are inspired by the way humans make decisions, comparing a test object to previously encountered samples. In this paper, we propose an NN algorithm that uses the lower and upper approximations from fuzzy-rough set theory in order to classify test objects, or predict their decision value. It is shown experimentally that our method outperforms other NN approaches (classical, fuzzy and fuzzy-rough ones) and that it is competitive with leading classification and prediction methods. Moreover, we show that the robustness of our methods against noise can be enhanced effectively by invoking the approximations of the Vaguely Quantified Rough Set (VQRS) model, which emulates the linguistic quantifiers ''some'' and ''most'' from natural language.

[1]  Qiang Shen,et al.  Interval-valued fuzzy-rough feature selection in datasets with missing values , 2009, 2009 IEEE International Conference on Fuzzy Systems.

[2]  Tzung-Pei Hong,et al.  Fuzzy rough sets with hierarchical quantitative attributes , 2009, Expert Syst. Appl..

[3]  Wojciech Ziarko,et al.  Variable Precision Rough Set Model , 1993, J. Comput. Syst. Sci..

[4]  Yong Wang.  A New Approach to Fitting Linear Models in High Dimensional Spaces , 2000 .

[5]  Nan-Chen Hsieh.  Rule Extraction with Rough-Fuzzy Hybridization Method , 2008, PAKDD.

[6]  Qiang Shen,et al.  A rough-fuzzy approach for generating classification rules , 2002, Pattern Recognit..

[7]  Martine De Cock,et al.  On (un)suitable fuzzy relations to model approximate equality , 2003, Fuzzy Sets Syst..

[8]  A. L. Edwards,et al.  An introduction to linear regression and correlation. , 1985 .

[9]  P. Viswanath,et al.  Rough-fuzzy weighted k-nearest leader classifier for large data sets , 2009, Pattern Recognit..

[10]  Chris Cornelis,et al.  A New Approach to Fuzzy-Rough Nearest Neighbour Classification , 2008, RSCTC.

[11]  Chris Cornelis,et al.  A noise-tolerant approach to fuzzy-rough feature selection , 2008, 2008 IEEE International Conference on Fuzzy Systems (IEEE World Congress on Computational Intelligence).

[12]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[13]  P. Langley.  Selection of Relevant Features in Machine Learning , 1994 .

[14]  Jerzy W. Grzymala-Busse,et al.  Three discretization methods for rule induction , 2001, Int. J. Intell. Syst..

[15]  Manish Sarkar,et al.  Fuzzy-rough nearest neighbors algorithm , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[16]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[17]  Rajen B. Bhatt,et al.  FRID: fuzzy-rough interactive dichotomizers , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[18]  Chen Li,et al.  A Fast and Scalable Fuzzy-rough Nearest Neighbor Algorithm , 2009, 2009 WRI Global Congress on Intelligent Systems.

[19]  Janusz Zalewski,et al.  Rough sets: Theoretical aspects of reasoning about data , 1996 .

[20]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[21]  Ian H. Witten,et al.  Generating Accurate Rule Sets Without Global Optimization , 1998, ICML.

[22]  Wojciech Ziarko,et al.  Decision Making with Probabilistic Decision Tables , 1999, RSFDGrC.

[23]  Patrick Brézillon,et al.  Lecture Notes in Artificial Intelligence , 1999 .

[24]  Xiangyang Wang,et al.  Fuzzy-Rough Set Based Nearest Neighbor Clustering Classification Algorithm , 2005, FSKD.

[25]  Michio Sugeno,et al.  Fuzzy identification of systems and its applications to modeling and control , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[26]  Masahiro Inuiguchi,et al.  Fuzzy rough sets and multiple-premise gradual decision rules , 2006, Int. J. Approx. Reason..

[27]  Tsau Young Lin,et al.  Granular Computing , 2003, RSFDGrC.

[28]  H. Bian,et al.  Fuzzy-rough nearest-neighbor classification approach , 2003, 22nd International Conference of the North American Fuzzy Information Processing Society, NAFIPS 2003.

[29]  Wojciech Ziarko.  Set Approximation Quality Measures in the Variable Precision Rough Set Model , 2002, HIS.

[30]  Witold Pedrycz,et al.  Granular computing: an introduction , 2001, Proceedings Joint 9th IFSA World Congress and 20th NAFIPS International Conference (Cat. No. 01TH8569).

[31]  Xizhao Wang,et al.  Learning fuzzy rules from fuzzy samples based on rough set technique , 2007, Inf. Sci..

[32]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[33]  Lotfi A. Zadeh,et al.  A COMPUTATIONAL APPROACH TO FUZZY QUANTIFIERS IN NATURAL LANGUAGES , 1983 .

[34]  D. Dubois,et al.  ROUGH FUZZY SETS AND FUZZY ROUGH SETS , 1990 .

[35]  Stephen Slade,et al.  Case-Based Reasoning: A Research Paradigm , 1991, AI Mag..

[36]  C. Cornelis,et al.  Vaguely Quantified Rough Sets , 2009, RSFDGrC.

[37]  James M. Keller,et al.  A fuzzy K-nearest neighbor algorithm , 1985, IEEE Transactions on Systems, Man, and Cybernetics.

[38]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[39]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[40]  Andrzej Skowron,et al.  Rough-Fuzzy Hybridization: A New Trend in Decision Making , 1999 .

[41]  Lotfi A. Zadeh,et al.  Soft computing and fuzzy logic , 1994, IEEE Software.

[42]  Hung Son Nguyen,et al.  Discretization Problem for Rough Sets Methods , 1998, Rough Sets and Current Trends in Computing.

[43]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[44]  Chris Cornelis,et al.  Attribute selection with fuzzy decision reducts , 2010, Inf. Sci..

[45]  Anna Maria Radzikowska,et al.  A comparative study of fuzzy rough sets , 2002, Fuzzy Sets Syst..

[46]  Manish Sarkar,et al.  Fuzzy-rough nearest neighbor algorithms in classification , 2007, Fuzzy Sets Syst..

[47]  Qiang Shen,et al.  New Approaches to Fuzzy-Rough Feature Selection , 2009, IEEE Transactions on Fuzzy Systems.

[48]  Javier Ruiz-del-Solar,et al.  Soft computing systems : design, management and applications , 2002 .

[49]  Qiang Shen,et al.  Computational Intelligence and Feature Selection - Rough and Fuzzy Approaches , 2008, IEEE Press series on computational intelligence.