A Generalized Kernel Approach to Dissimilarity-based Classification

Usually, objects to be classified are represented by features. In this paper, we discuss an alternative object representation based on dissimilarity values. If such distances separate the classes well, the nearest neighbor method offers a good solution. However, dissimilarities used in practice are usually far from ideal and the performance of the nearest neighbor rule suffers from its sensitivity to noisy examples. We show that other, more global classification techniques are preferable to the nearest neighbor rule, in such cases.For classification purposes, two different ways of using generalized dissimilarity kernels are considered. In the first one, distances are isometrically embedded in a pseudo-Euclidean space and the classification task is performed there. In the second approach, classifiers are built directly on distance kernels. Both approaches are described theoretically and then compared using experiments with different dissimilarity measures and datasets including degraded data simulating the problem of missing values.

[1]  A. Householder,et al.  Discussion of a set of points in terms of their mutual distances , 1938 .

[2]  A. G. Arkad'ev,et al.  Computers and pattern recognition , 1967 .

[3]  N. JARDINE,et al.  A New Approach to Pattern Recognition , 1971, Nature.

[4]  David G. Stork,et al.  Pattern Classification , 1973 .

[5]  A. Tversky Features of Similarity , 1977 .

[6]  J. Gower Euclidean Distance Geometry , 1982 .

[7]  Lev Goldfarb,et al.  A unified approach to pattern recognition , 1984, Pattern Recognit..

[8]  J. Gower,et al.  Metric and Euclidean properties of dissimilarity coefficients , 1986 .

[9]  Keinosuke Fukunaga,et al.  Introduction to statistical pattern recognition (2nd ed.) , 1990 .

[10]  O. Mangasarian,et al.  Robust linear programming discrimination of two linearly inseparable sets , 1992 .

[11]  Anil K. Jain,et al.  A modified Hausdorff distance for object matching , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[12]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[13]  Bernhard Schölkopf,et al.  Kernel Principal Component Analysis , 1997, ICANN.

[14]  Bernhard Schölkopf,et al.  Support vector learning , 1997 .

[15]  Anil K. Jain,et al.  Representation and Recognition of Handwritten Digits Using Deformable Templates , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  R. Duin,et al.  COMPLEXITY OF DISSIMILARITY BASED PATTERN CLASSES , 1998 .

[17]  Bernhard Schölkopf,et al.  Semiparametric Support Vector and Linear Programming Machines , 1998, NIPS.

[18]  Paul S. Bradley,et al.  Feature Selection via Mathematical Programming , 1997, INFORMS J. Comput..

[19]  Klaus Obermayer,et al.  Classi cation on Pairwise Proximity , 2007 .

[20]  R. C. Williamson,et al.  Classification on proximity data with LP-machines , 1999 .

[21]  Christopher J. C. Burges,et al.  Geometry and invariance in kernel based methods , 1999 .

[22]  Robert P. W. Duin,et al.  Relational discriminant analysis , 1999, Pattern Recognit. Lett..

[23]  G. Wahba Support vector machines, reproducing kernel Hilbert spaces, and randomized GACV , 1999 .

[24]  Daphna Weinshall,et al.  Classification with Nonmetric Distances: Image Retrieval and Class Representation , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Robert P. W. Duin,et al.  Classifiers in almost empty spaces , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[28]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[29]  Philip N. Klein,et al.  Recognition of Shapes by Editing Shock Graphs , 2001, ICCV.

[30]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[31]  R. Duin,et al.  Automatic pattern recognition by similarity representations , 2001 .