An instance-based learning approach based on grey relational structure

In instance-based learning, the ‘nearness’ between two instances—used for pattern classification—is generally determined by some similarity functions, such as the Euclidean or Value Difference Metric (VDM). However, Euclidean-like similarity functions are normally only suitable for domains with numeric attributes. The VDM metrics are mainly applicable to domains with symbolic attributes, and their complexity increases with the number of classes in a specific application domain. This paper proposes an instance-based learning approach to alleviate these shortcomings. Grey relational analysis is used to precisely describe the entire relational structure of all instances in a specific domain. By using the grey relational structure, new instances can be classified with high accuracy. Moreover, the total number of classes in a specific domain does not affect the complexity of the proposed approach. Forty classification problems are used for performance comparison. Experimental results show that the proposed approach yields higher performance over other methods that adopt one of the above similarity functions or both. Meanwhile, the proposed method can yield higher performance, compared to some other classification algorithms.

[1]  James P. Ignizio,et al.  Simultaneous design and training of ontogenic neural network classifiers , 1996, Comput. Oper. Res..

[2]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[3]  J. Deng,et al.  Introduction to Grey system theory , 1989 .

[4]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[5]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[6]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques with Java implementations , 2002, SGMD.

[7]  David W. Aha,et al.  Towards a Better Understanding of Memory-based Reasoning Systems , 1994, ICML.

[8]  David W. Aha,et al.  Learning Representative Exemplars of Concepts: An Initial Case Study , 1987 .

[9]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[10]  Yi-Chung Hu,et al.  Grey self-organizing feature maps , 2002, Neurocomputing.

[11]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[12]  Herbert A. Simon,et al.  Applications of machine learning and rule induction , 1995, CACM.

[13]  Ramin Yasdi,et al.  Learning Classification Rules from Database in the Context of Knowledge Acquisition and Representation , 1991, IEEE Trans. Knowl. Data Eng..

[14]  Chi-Chun Huang,et al.  A Novel Partial-Memory Learning Algorithm Based on Grey Relational Structure , 2003, IDA.

[15]  Rattikorn Hewett,et al.  Restructuring decision tables for elucidation of knowledge , 2003, Data Knowl. Eng..

[16]  Robert C. Holte,et al.  Very Simple Classification Rules Perform Well on Most Commonly Used Datasets , 1993, Machine Learning.

[17]  Collin J. Watson,et al.  Statistics for Management and Economics , 1991 .

[18]  Kazuo Hattori,et al.  A new edited k-nearest neighbor rule in the pattern classification problem , 2000, Pattern Recognit..

[19]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[20]  Aijun An.  Learning classification rules from data , 2003 .

[21]  J. Ross Quinlan,et al.  Induction of Decision Trees , 1986, Machine Learning.

[22]  Tony R. Martinez,et al.  Reduction Techniques for Instance-Based Learning Algorithms , 2000, Machine Learning.

[23]  Shusaku Tsumoto,et al.  Automated extraction of hierarchical decision rules from clinical databases using rough set model , 2003, Expert Syst. Appl..

[24]  David W. Aha,et al.  Tolerating Noisy, Irrelevant and Novel Attributes in Instance-Based Learning Algorithms , 1992, Int. J. Man Mach. Stud..

[25]  Ralph Martinez,et al.  Reduction Techniques for Exemplar-Based Learning Algorithms , 1998 .

[26]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[27]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .

[28]  Ray J. Hickey,et al.  An instance-based approach to pattern association learning with application to the English past tense verb domain , 2001, Knowl. Based Syst..

[29]  Chi-Chun Huang,et al.  A Grey-Based Nearest Neighbor Approach for Missing Attribute Value Prediction , 2004, Applied Intelligence.

[30]  Ron Kohavi,et al.  The Power of Decision Tables , 1995, ECML.

[31]  Yoav Freund,et al.  The Alternating Decision Tree Learning Algorithm , 1999, ICML.

[32]  Jerome H. Friedman,et al.  A Recursive Partitioning Decision Rule for Nonparametric Classification , 1977, IEEE Transactions on Computers.

[33]  Yo-Ping Huang,et al.  Real-valued genetic algorithms for fuzzy grey prediction system , 1997, Fuzzy Sets Syst..

[34]  J. L. Hodges,et al.  Discriminatory Analysis - Nonparametric Discrimination: Consistency Properties , 1989 .

[35]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[36]  Gerald Keller,et al.  Statistics for Management and Economics , 1990 .

[37]  Richard O. Mason,et al.  Applying ethics to information technology issues , 1995, CACM.

[38]  Ian D. Watson,et al.  Case-based reasoning is a methodology not a technology , 1999, Knowl. Based Syst..

[39]  Eyke Hüllermeier,et al.  Possibilistic instance-based learning , 2003, Artif. Intell..

[40]  Roelof K. Brouwer.  Automatic Growing of a Hopfield Style Network During Training for Classification , 1997, Neural Networks.

[41]  Stephen D. Bay.  Nearest neighbor classification from multiple feature subsets , 1999, Intell. Data Anal..

[42]  Khaled Mellouli,et al.  Belief decision trees: theoretical foundations , 2001, Int. J. Approx. Reason..