Re-KISSME: A robust resampling scheme for distance metric learning in the presence of label noise

Abstract Distance metric learning aims to learn a metric with the similarity of samples. However, the increasing scalability and complexity of dataset or complex application brings about inevitable label noise, which frustrates the distance metric learning. In this paper, we propose a resampling scheme robust to label noise, Re-KISSME, based on Keep It Simple and Straightforward Metric (KISSME) learning method. Specifically, we consider the data structure and the priors of labels as two resampling factors to correct the observed distribution. By introducing the true similarity as latent variable, these two factors are integrated into a maximum likelihood estimation model. As a result, Re-KISSME can reason the underlying similarity of each pair and reduce the influence of label noise to estimate the metric matrix. Our model is solved by iterative algorithm with low computational cost. With synthetic label noise, the experiments on UCI datasets and two application datasets of person re-identification confirm the effectiveness of our proposal.

[1]  Cordelia Schmid,et al.  Is that you? Metric learning approaches for face identification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Ata Kabán,et al.  Label-Noise Robust Logistic Regression and Its Applications , 2012, ECML/PKDD.

[3]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[4]  Wei Liu,et al.  Learning Distance Metrics with Contextual Constraints for Image Retrieval , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[5]  Marcel J. T. Reinders,et al.  Classification in the presence of class noise using a probabilistic Kernel Fisher method , 2007, Pattern Recognit..

[6]  Abir Das,et al.  Consistent Re-identification in a Camera Network , 2014, ECCV.

[7]  Xuelong Li,et al.  Person Reidentification by Minimum Classification Error-Based KISS Metric Learning , 2015, IEEE Transactions on Cybernetics.

[8]  Gert R. G. Lanckriet,et al.  Metric Learning to Rank , 2010, ICML.

[9]  Peng Li,et al.  Distance Metric Learning with Eigenvalue Optimization , 2012, J. Mach. Learn. Res..

[10]  Aritra Ghosh,et al.  Robust Loss Functions under Label Noise for Deep Neural Networks , 2017, AAAI.

[11]  Shuicheng Yan,et al.  Graph Embedding and Extensions: A General Framework for Dimensionality Reduction , 2007 .

[12]  Gerardo Hermosillo,et al.  Learning From Crowds , 2010, J. Mach. Learn. Res..

[13]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Use of Classification Algorithms in Noise Detection and Elimination , 2009, HAIS.

[14]  Fei Xiong,et al.  Person Re-Identification Using Kernel-Based Metric Learning Methods , 2014, ECCV.

[15]  Cordelia Schmid,et al.  TagProp: Discriminative metric learning in nearest neighbor models for image auto-annotation , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Nagarajan Natarajan,et al.  Learning with Noisy Labels , 2013, NIPS.

[17]  Shengcai Liao,et al.  Person re-identification by Local Maximal Occurrence representation and metric learning , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[19]  Ata Kabán,et al.  Boosting in the presence of label noise , 2013, UAI.

[20]  Max A. Little,et al.  Exploiting Nonlinear Recurrence and Fractal Scaling Properties for Voice Disorder Detection , 2007, Biomedical engineering online.

[21]  Lu Wang,et al.  Risk Minimization in the Presence of Label Noise , 2016, AAAI.

[22]  Isabelle Guyon,et al.  Discovering Informative Patterns and Data Cleaning , 1996, Advances in Knowledge Discovery and Data Mining.

[23]  Shaogang Gong,et al.  Reidentification by Relative Distance Comparison , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Saso Dzeroski,et al.  Noise detection and elimination in data preprocessing: Experiments in medical domains , 2000, Appl. Artif. Intell..

[26]  Yuan Yan Tang,et al.  Person Re-Identification by Dual-Regularized KISS Metric Learning , 2016, IEEE Transactions on Image Processing.

[27]  Feiping Nie,et al.  Learning a Mahalanobis distance metric for data clustering and classification , 2008, Pattern Recognit..

[28]  Zhi-Hua Zhou,et al.  What Makes Objects Similar: A Unified Multi-Metric Learning Approach , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Tomer Hertz,et al.  Learning Distance Functions using Equivalence Relations , 2003, ICML.

[30]  Brian Kulis,et al.  Metric Learning: A Survey , 2013, Found. Trends Mach. Learn..

[31]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[32]  Dacheng Tao,et al.  Classification with Noisy Labels by Importance Reweighting , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Wensheng Zhang,et al.  A method for predicting disease subtypes in presence of misclassification among training samples using gene expression: application to human breast cancer , 2006, Bioinform..

[34]  Rong Jin,et al.  Learning from Noisy Side Information by Generalized Maximum Entropy Model , 2010, ICML.

[35]  Shaogang Gong,et al.  Associating Groups of People , 2009, BMVC.

[36]  Fabrice Muhlenbach,et al.  Identifying and Handling Mislabelled Instances , 2004, Journal of Intelligent Information Systems.

[37]  Xiaoyang Tan,et al.  Robust Distance Metric Learning in the Presence of Label Noise , 2014, AAAI.

[38]  Zhen Li,et al.  Learning Locally-Adaptive Decision Functions for Person Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  M. Verleysen,et al.  Classification in the Presence of Label Noise: A Survey , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Carla E. Brodley,et al.  Identifying Mislabeled Training Data , 1999, J. Artif. Intell. Res..

[41]  Alexandros Kalousis,et al.  Parametric Local Metric Learning for Nearest Neighbor Classification , 2012, NIPS.

[42]  Jinfeng Yi,et al.  Semi-Crowdsourced Clustering: Generalizing Crowd Labeling by Robust Distance Metric Learning , 2012, NIPS.

[43]  Frédéric Jurie,et al.  PCCA: A new approach for distance learning from sparse pairwise constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Gang Hua,et al.  Discriminative Tracking by Metric Learning , 2010, ECCV.

[45]  Stephen Tyree,et al.  Non-linear Metric Learning , 2012, NIPS.

[46]  M. Hestenes,et al.  Methods of conjugate gradients for solving linear systems , 1952 .