Independently weighted value difference metric

Abstract The majority of the difference metrics used in categorical classification algorithms do not take the dependence structure among attributes into account. Some of these metrics even make strong assumptions on attribute independence which are not realistic for many real-world datasets. In addition, these metrics do not consider attribute importance on the class variable. In this paper, a new difference metric is proposed which is named as Independently Weighted Value Difference Metric (IWVDM). IWVDM includes an embedded Incremental Feature Selection (IFS) phase. The proposed metric does not require attribute independence and it introduces a weighting procedure for attributes depending on the information that they possess on the class variable. A series of experiments is conducted using 30 UCI benchmark datasets for comparing the efficiency of IWVDM with Overlap Metric (OM), Value Difference Metric (VDM) and Frequency Difference Metric (FDM). Experimental results show the superiority of IWVDM over these three metrics.

[1]  David W. Aha,et al.  A Probabilistic Framework for Memory-Based Reasoning , 1998, Artif. Intell..

[2]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[3]  Hongwei Li,et al.  One Dependence Value Difference Metric , 2011, Knowl. Based Syst..

[4]  Zaher Al,et al.  Classification of Categorical and Numerical Data on Selected Subset of Features , 2010 .

[5]  Liangxiao Jiang,et al.  A Novel Distance Function: frequency difference Metric , 2014, Int. J. Pattern Recognit. Artif. Intell..

[6]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[7]  Te Sun Han,et al.  Multiple Mutual Informations and Multiple Interactions in Frequency Data , 1980, Inf. Control..

[8]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[9]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[10]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[11]  Liangxiao Jiang,et al.  An Augmented Value Difference Measure , 2013, Pattern Recognit. Lett..

[12]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[13]  S. Archana,et al.  Survey of Classification Techniques in Data Mining , 2014 .

[14]  Jonathan Goldstein,et al.  When Is ''Nearest Neighbor'' Meaningful? , 1999, ICDT.

[15]  Liangxiao Jiang,et al.  An Empirical Study on Class Probability Estimates in Decision Tree Learning , 2011, J. Softw..

[16]  Thomas M. Cover,et al.  Elements of Information Theory: Cover/Elements of Information Theory, Second Edition , 2005 .

[17]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[18]  David R. Brillinger,et al.  Some data analyses using mutual information , 2004 .

[19]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[20]  Diana Sommer,et al.  Log Linear Models And Logistic Regression , 2016 .

[21]  D. Bauer.  Constructing Confidence Sets Using Rank Statistics , 1972 .

[22]  Daniel A. Keim,et al.  Optimal Grid-Clustering: Towards Breaking the Curse of Dimensionality in High-Dimensional Clustering , 1999, VLDB.

[23]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[24]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[25]  Huan Liu,et al.  Feature Selection for High-Dimensional Data: A Fast Correlation-Based Filter Solution , 2003, ICML.

[26]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[27]  S. Wood.  Fast stable restricted maximum likelihood and marginal likelihood estimation of semiparametric generalized linear models , 2011 .

[28]  Vivek Agarwal,et al.  Survey on Classification Techniques for Data Mining , 2015 .

[29]  David W. Aha,et al.  Tolerating Noisy, Irrelevant and Novel Attributes in Instance-Based Learning Algorithms , 1992, Int. J. Man Mach. Stud..

[30]  Shasha Wang,et al.  Attribute Weighted Value Difference Metric , 2013, 2013 IEEE 25th International Conference on Tools with Artificial Intelligence.

[31]  Sunil Srinivasa.  A REVIEW ON MULTIVARIATE MUTUAL INFORMATION , 2005 .