A Study on Combining Sets of Differently Measured Dissimilarities

The ways distances are computed or measured enable us to have different representations of the same objects. In this paper we want to discuss possible ways of merging different sources of information given by differently measured dissimilarity representations. We compare here a simple averaging scheme [1] with dissimilarity forward selection and other techniques based on the learning of weights of linear and quadratic forms. Our general conclusion is that, although the more advanced forms of combination cannot always lead to better classification accuracies, combining given distance matrices prior to training is always worthwhile. We can thereby suggest which combination schemes are preferable with respect to the problem data.

[1]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[2]  Mordecai Avriel,et al.  Nonlinear programming , 1976 .

[3]  Horst Bunke,et al.  Applications of approximate string matching to 2D shape recognition , 1993, Pattern Recognit..

[4]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[5]  N. Cristianini,et al.  On Kernel-Target Alignment , 2001, NIPS.

[6]  Robert P. W. Duin,et al.  On Combining Dissimilarity Representations , 2001, Multiple Classifier Systems.

[7]  Nello Cristianini,et al.  Learning the Kernel Matrix with Semidefinite Programming , 2002, J. Mach. Learn. Res..

[8]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[9]  Javier M. Moguerza,et al.  Combining Kernel Information for Support Vector Classification , 2004, Multiple Classifier Systems.

[10]  Robert P. W. Duin,et al.  Combining Dissimilarity-Based One-Class Classifiers , 2004, Multiple Classifier Systems.

[11]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[12]  Amir Globerson,et al.  Metric Learning by Collapsing Classes , 2005, NIPS.

[13]  Yi Liu,et al.  An Efficient Algorithm for Local Distance Metric Learning , 2006, AAAI.

[14]  Wan-Jui Lee,et al.  Kernel Combination Versus Classifier Combination , 2007, MCS.

[15]  Melanie Hilario,et al.  Learning to combine distances for complex representations , 2007, ICML '07.

[16]  R. Duin,et al.  A Multiscale Approach in Combining Classifiers in Dissimilarity Representations , 2009 .

[17]  Marcel J. T. Reinders,et al.  Evolutionary Optimization of Kernel Weights Improves Protein Complex Comembership Prediction , 2009, IEEE/ACM Transactions on Computational Biology & Bioinformatics.

[18]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.