Robust Distance Metric Learning via Bayesian Inference

Distance metric learning (DML) has achieved great success in many computer vision tasks. However, most existing DML algorithms are based on point estimation, and thus are sensitive to the choice of training examples and tend to be over-fitting in the presence of label noise. In this paper, we present a robust DML algorithm based on Bayesian inference. In particular, our method is essentially a Bayesian extension to a previous classic DML method—large margin nearest neighbor classification and we use stochastic variational inference to estimate the posterior distribution of the transformation matrix. Furthermore, we theoretically show that the proposed algorithm is robust against label noise in the sense that an arbitrary point with label noise has bounded influence on the learnt model. With some reasonable assumptions, we derive a generalization error bound of this method in the presence of label noise. We also show that the DML hypothesis class in which our model lies is probably approximately correct-learnable and give the sample complexity. The effectiveness of the proposed method1 is demonstrated with state of the art performance on three popular data sets with different types of label noise.1A MATLAB implementation of this method is made available at http://parnec.nuaa.edu.cn/xtan/Publication.htm

[1]  Chong Wang,et al.  Stochastic variational inference , 2012, J. Mach. Learn. Res..

[2]  Jiwen Lu,et al.  Learning a Discriminative Distance Metric With Label Consistency for Scene Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[3]  André Elisseeff,et al.  Stability and Generalization , 2002, J. Mach. Learn. Res..

[4]  Rémi Emonet,et al.  Metric Learning as Convex Combinations of Local Models with Generalization Guarantees , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[6]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[7]  Rong Jin,et al.  Regularized Distance Metric Learning: Theory and Algorithm , 2009, NIPS.

[8]  Rong Jin,et al.  Bayesian Active Distance Metric Learning , 2007, UAI.

[9]  Feiping Nie,et al.  Robust and Effective Metric Learning Using Capped Trace Norm: Metric Learning via Capped Trace Norm , 2016, KDD.

[10]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[11]  Prateek Jain,et al.  On the Generalization Ability of Online Learning Algorithms for Pairwise Loss Functions , 2013, ICML.

[12]  Eyal Kushilevitz,et al.  PAC learning with nasty noise , 1999, Theor. Comput. Sci..

[13]  Shengcai Liao,et al.  Large Scale Similarity Learning Using Similar Pairs for Person Verification , 2016, AAAI.

[14]  Arif Mahmood,et al.  Constrained Metric Learning by Permutation Inducing Isometries , 2016, IEEE Transactions on Image Processing.

[15]  Jun Zhu,et al.  Max-Margin Nonparametric Latent Feature Models for Link Prediction , 2012, ICML.

[16]  Jian Pei,et al.  Distance metric learning using dropout: a structured regularization approach , 2014, KDD.

[17]  Feiping Nie,et al.  Robust Distance Metric Learning via Simultaneous L1-Norm Minimization and Maximization , 2014, ICML.

[18]  Dapeng Tao,et al.  Person Re-Identification by Dual-Regularized KISS Metric Learning. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[19]  Nicholas G. Polson,et al.  Data augmentation for support vector machines , 2011 .

[20]  Suvrit Sra,et al.  Geometric Mean Metric Learning , 2016, ICML.

[21]  David Ruppert,et al.  Tapered Covariance: Bayesian Estimation and Asymptotics , 2012 .

[22]  Michael S. Bernstein,et al.  Embracing Error to Enable Rapid Crowdsourcing , 2016, CHI.

[23]  Baba C. Vemuri,et al.  A Robust and Efficient Doubly Regularized Metric Learning Approach , 2012, ECCV.

[24]  Heng Tao Shen,et al.  Principal Component Analysis , 2009, Encyclopedia of Biometrics.

[25]  Gabriela Csurka,et al.  Metric Learning for Large Scale Image Classification: Generalizing to New Classes at Near-Zero Cost , 2012, ECCV.

[26]  Xiaoyang Tan,et al.  Unsupervised feature learning with C-SVDDNet , 2014, Pattern Recognit..

[27]  Bernhard Schölkopf,et al.  Estimating a Kernel Fisher Discriminant in the Presence of Label Noise , 2001, ICML.

[28]  M. Verleysen,et al.  Classification in the Presence of Label Noise: A Survey , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[29]  Geoffrey E. Hinton,et al.  Neighbourhood Components Analysis , 2004, NIPS.

[30]  Lu Wang,et al.  Risk Minimization in the Presence of Label Noise , 2016, AAAI.

[31]  Andrew Zisserman,et al.  Deep Face Recognition , 2015, BMVC.

[32]  Javed A. Aslam,et al.  On the Sample Complexity of Noise-Tolerant Learning , 1996, Inf. Process. Lett..

[33]  Jun Zhu,et al.  Bayesian Max-margin Multi-Task Learning with Data Augmentation , 2014, ICML.

[34]  Yunhong Wang,et al.  Relevance Metric Learning for Person Re-Identification by Exploiting Listwise Similarities , 2015, IEEE Transactions on Image Processing.

[35]  Marc Sebban,et al.  Similarity Learning for Provably Accurate Sparse Linear Classification , 2012, ICML.

[36]  Matthieu Cord,et al.  Fantope Regularization in Metric Learning , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Ning Chen,et al.  Bayesian inference with posterior regularization and applications to infinite latent SVMs , 2012, J. Mach. Learn. Res..

[38]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Yuan Yan Tang,et al.  Person Re-Identification by Dual-Regularized KISS Metric Learning , 2016, IEEE Transactions on Image Processing.

[40]  Yuxiao Hu,et al.  MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition , 2016, ECCV.

[41]  Nagarajan Natarajan,et al.  Learning with Noisy Labels , 2013, NIPS.

[42]  Sandra Zilles,et al.  PAC-Learning with General Class Noise Models , 2012, KI.

[43]  Yun Fu,et al.  Robust Transfer Metric Learning for Image Classification , 2017, IEEE Transactions on Image Processing.

[44]  Qiong Cao,et al.  Generalization bounds for metric and similarity learning , 2012, Machine Learning.

[45]  Huchuan Lu,et al.  Person Re-Identification via Distance Metric Learning With Latent Variables , 2017, IEEE Transactions on Image Processing.

[46]  Xiaoyang Tan,et al.  Robust Distance Metric Learning in the Presence of Label Noise , 2014, AAAI.

[47]  Wenbin Yao,et al.  Diversity regularized metric learning for person re-identification , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[48]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[49]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[50]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.