Specific-class distance measures for nominal attributes

The classification accuracy of many machine learning methods depends upon their ability to accurately measure the similarity between different instances. Similarity is measured using a distance metric or measure. In this work, several novel distance measures for nominal values are proposed. These distance measures exploit the class of a training example against which a new instance is compared. The experiments, conducted using 50 benchmark data sets, indicate that the proposed functions are superior in many cases to the Value Difference Metric VDM that is widely used in instance based learning. Some of the proposed measures have proven to be less sensitive to missing values and noise in the training data sets and have maintained good classification accuracy in the presence of unknown and noisy attribute values. Like VDM, the proposed measures work only with labelled training data sets which makes them unsuitable for unsupervised learning methods.

[1]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[2]  Mousa Al-Akhras,et al.  Smoothing decision boundaries to avoid overfitting in neural network training , 2011 .

[3]  Hui Wang,et al.  Nearest neighbors by neighborhood counting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Werner Dubitzky,et al.  A flexible and robust similarity measure based on contextual probability , 2005, IJCAI.

[5]  Francesco Ricci,et al.  Probability Based Metrics for Nearest Neighbor Classification and Case-Based Reasoning , 1999, ICCBR.

[6]  David L. Waltz,et al.  Toward memory-based reasoning , 1986, CACM.

[7]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[8]  Ian Witten,et al.  Data Mining , 2000 .

[9]  Martin D. Buhmann,et al.  Radial Basis Functions , 2021, Encyclopedia of Mathematical Geosciences.

[10]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[11]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[12]  Hui Wang.  Neighborhood Counting Measure and Minimum Risk Metric , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Amir Globerson,et al.  Metric Learning by Collapsing Classes , 2005, NIPS.

[14]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[15]  Cullen Schaffer,et al.  A Conservation Law for Generalization Performance , 1994, ICML.

[16]  Usama M. Fayyad,et al.  Multi-Interval Discretization of Continuous-Valued Attributes for Classification Learning , 1993, IJCAI.

[17]  Dennis L. Wilson,et al.  Asymptotic Properties of Nearest Neighbor Rules Using Edited Data , 1972, IEEE Trans. Syst. Man Cybern..

[18]  Ron Kohavi,et al.  Supervised and Unsupervised Discretization of Continuous Features , 1995, ICML.

[19]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[20]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[21]  Marc Boullé,et al.  MODL: A Bayes optimal discretization method for continuous attributes , 2006, Machine Learning.