Semi-supervised Coefficient-Based Distance Metric Learning

Distance metric learning plays an important role in real-world applications, such as image classification and clustering. Previous works mainly learn a distance metric through learning a Mahalanobis metric or learning a linear transformation. In this paper, we propose to learn a distance metric from a new perspective. We first randomly generate a set of base vectors and then learn a linear combination of these vectors to approximate the target metric. Compared with previous distance metric learning methods, we only need to learn the coefficients of these base vectors instead of learning the target metric or the linear transformation. Consequently, the number of variables needed to be determined is the same as the number of base vectors, which is irrelevant to the dimension of the data. Furthermore, considering the situation that labeled samples are insufficient in some cases, we extend our proposed distance metric learning method into a semi-supervised learning framework. Additionally, an optimization algorithm is proposed to accelerate training of our proposed methods. Experiments are conducted on several datasets and the results demonstrate the effectiveness of our proposed methods.

[1]  Yong Luo,et al.  Decomposition-Based Transfer Distance Metric Learning for Image Classification , 2014, IEEE Transactions on Image Processing.

[2]  Tomer Hertz,et al.  Learning a Mahalanobis Metric from Equivalence Constraints , 2005, J. Mach. Learn. Res..

[3]  Xindong Wu,et al.  NESVM: A Fast Gradient Method for Support Vector Machines , 2010, 2010 IEEE International Conference on Data Mining.

[4]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[5]  Mahdieh Soleymani Baghshah,et al.  Semi-Supervised Metric Learning Using Pairwise Constraints , 2009, IJCAI.

[6]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[7]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[8]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[9]  Wei Liu,et al.  Semi-supervised distance metric learning for Collaborative Image Retrieval , 2008, CVPR.

[10]  Wei Liu,et al.  Learning Distance Metrics with Contextual Constraints for Image Retrieval , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[11]  Rong Jin,et al.  Regularized Distance Metric Learning: Theory and Algorithm , 2009, NIPS.

[12]  Meng Wang,et al.  Semisupervised Multiview Distance Metric Learning for Cartoon Synthesis , 2012, IEEE Transactions on Image Processing.

[13]  Li Ya,et al.  Regularized Large Margin Distance Metric Learning , 2016 .

[14]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[15]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.