Online and Batch Learning of Generalized Cosine Similarities

In this paper, we define an online algorithm to learn the generalized cosine similarity measures for kNN classification and hence a similarity matrix A corresponding to a bilinear form. In contrary to the standard cosine measure, the normalization is itself dependent on the similarity matrix which makes it impossible to use directly the algorithms developed for learning Mahanalobis distances, based on positive, semi-definite (PSD) matrices. We follow the approach where we first find an appropriate matrix and then project it onto the cone of PSD matrices, which we have adapted to the particular form of generalized cosine similarities, and more particularly to the fact that such measures are normalized. The resulting online algorithm as well as its batch version is fast and has got better accuracy as compared with state-of-the-art methods on standard data sets.

[1]  Michael L. Raymer,et al.  GA-facilitated KNN classifier optimization with varying similarity measures , 2005, 2005 IEEE Congress on Evolutionary Computation.

[2]  Joo-Hwee Lim,et al.  Similarity Learning for Nearest Neighbor Classification , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[3]  Yoram Singer,et al.  Online and batch learning of pseudo-metrics , 2004, ICML.

[4]  Yoram Singer,et al.  Data-Driven Online to Batch Conversions , 2005, NIPS.

[5]  Maya R. Gupta,et al.  Similarity-based Classification: Concepts and Algorithms , 2009, J. Mach. Learn. Res..

[6]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT' 98.

[7]  Andrzej Szalas,et al.  A Technique for Learning Similarities on Complex Structures with Applications to Extracting Ontologies , 2005, AWIC.

[8]  Marco Maggini,et al.  Learning Similarities for Text Documents using Neural Networks , 2003 .

[9]  Amir Globerson,et al.  Metric Learning by Collapsing Classes , 2005, NIPS.

[10]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[11]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[12]  Yoav Freund,et al.  Large Margin Classification Using the Perceptron Algorithm , 1998, COLT.

[13]  Marco Maggini,et al.  A Neural Network Approach to Similarity Learning , 2008, ANNPR.

[14]  Jun-Yi Shen,et al.  Quick asymmetric text similarity measures , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[15]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[16]  Shiwen Yu,et al.  An adaptive k-nearest neighbor text categorization strategy , 2004, TALIP.