A nearest-neighbor search model for distance metric learning

Abstract Distance metric learning aims to deal with the data distribution by learning a suitable distance metric from the training instances. For distance metric learning, the optimization constraints can be constructed based on the similar and dissimilar instance pairs. The instance pairs are generated by selecting the nearest-neighbors for each training instance. However, most methods select the same and fixed nearest-neighbor number for different training instances, which may limit performance for learning distance metric. In this paper, we propose a nearest-neighbor search model for distance metric learning (NNS-DML), which is capable of constructing the metric optimization constraints by searching different optimal nearest-neighbor numbers for different training instances. Specifically, we formulate a nearest-neighbor search matrix to contain the nearest-neighbor correlations of all training instances. Using the search matrix, we can construct and weight the metric optimization constraints of each training instance, such that the influence of its irrelevant features for its corresponding similar and dissimilar instance pairs can be reduced. Moreover, we develop a k-free nearest-neighbor model for classification problems via the SVM solver, which can ignore the setting of k. Extensive experiments show that the proposed NNS-DML method outperforms the state-of-the-art distance metric learning methods.

[1]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Jiwen Lu,et al.  Discriminative Deep Metric Learning for Face and Kinship Verification , 2017, IEEE Transactions on Image Processing.

[3]  Dewei Li,et al.  Survey and experimental study on metric learning methods , 2018, Neural Networks.

[4]  Jundong Liu,et al.  Nonlinear Metric Learning for kNN and SVMs through Geometric Transformations , 2015, Neurocomputing.

[5]  Kathrin Klamroth,et al.  Biconvex sets and optimization with biconvex functions: a survey and extensions , 2007, Math. Methods Oper. Res..

[6]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[7]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[8]  Yaxin Peng,et al.  Performance Analysis for SVM Combining with Metric Learning , 2017, Neural Processing Letters.

[9]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[10]  Stan Sclaroff,et al.  Deep Metric Learning to Rank , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Dong Xu,et al.  Distance Metric Learning Using Privileged Information for Face Verification and Person Re-Identification , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[12]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[13]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[14]  Vladimir Vapnik,et al.  Support-vector networks , 2004, Machine Learning.

[15]  Zhijie Wen,et al.  Manifold Preserving: An Intrinsic Approach for Semisupervised Distance Metric Learning , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[16]  David Zhang,et al.  A Kernel Classification Framework for Metric Learning , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[17]  Chunyan Miao,et al.  Online Multi-Modal Distance Metric Learning with Application to Image Retrieval , 2016, IEEE Transactions on Knowledge and Data Engineering.

[18]  Bernard De Baets,et al.  An approach to supervised distance metric learning based on difference of convex functions programming , 2018, Pattern Recognit..

[19]  Alexander J. Smola,et al.  Online learning with kernels , 2001, IEEE Transactions on Signal Processing.

[20]  Brian Kulis,et al.  Metric Learning: A Survey , 2013, Found. Trends Mach. Learn..

[21]  Inderjit S. Dhillon,et al.  Metric and Kernel Learning Using a Linear Transformation , 2009, J. Mach. Learn. Res..

[22]  Jiwen Lu,et al.  Discriminative Deep Metric Learning for Face Verification in the Wild , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Hoel Le Capitaine,et al.  Constraint selection in metric learning , 2016, Knowl. Based Syst..

[24]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[25]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Bernard De Baets,et al.  Kernel Distance Metric Learning Using Pairwise Constraints for Person Re-Identification , 2019, IEEE Transactions on Image Processing.

[27]  David Zhang,et al.  Distance Metric Learning via Iterated Support Vector Machines , 2017, IEEE Transactions on Image Processing.

[28]  Ling Shao,et al.  Human Action Recognition Using LBP-TOP as Sparse Spatio-Temporal Feature Descriptor , 2009, CAIP.

[29]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[30]  Min Xu,et al.  Representing documents through their readers , 2013, KDD.

[31]  Yun Fu,et al.  SLMOML: Online Metric Learning With Global Convergence , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[32]  Xiaofeng Zhu,et al.  Efficient kNN Classification With Different Numbers of Nearest Neighbors , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Ichiro Takeuchi,et al.  Safe Triplet Screening for Distance Metric Learning , 2018, Neural Computation.

[34]  Wei Liu,et al.  Semi-supervised distance metric learning for collaborative image retrieval and clustering , 2010, ACM Trans. Multim. Comput. Commun. Appl..

[35]  Xuelong Li,et al.  Learning k for kNN Classification , 2017, ACM Trans. Intell. Syst. Technol..

[36]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[37]  Ioannis A. Kakadiaris,et al.  An Overview and Empirical Comparison of Distance Metric Learning Methods , 2017, IEEE Transactions on Cybernetics.

[38]  Matti Pietikäinen,et al.  Rotation-Invariant Image and Video Description With Local Binary Pattern Features , 2012, IEEE Transactions on Image Processing.

[39]  Guy Lebanon,et al.  Metric learning for text documents , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.