Learning Distance Metrics from Probabilistic Information

The goal of metric learning is to learn a good distance metric that can capture the relationships among instances, and its importance has long been recognized in many fields. An implicit assumption in the traditional settings of metric learning is that the associated labels of the instances are deterministic. However, in many real-world applications, the associated labels come naturally with probabilities instead of deterministic values, which makes it difficult for the existing metric-learning methods to work well in these applications. To address this challenge, in this article, we study how to effectively learn the distance metric from datasets that contain probabilistic information, and then propose several novel metric-learning mechanisms for two types of probabilistic labels, i.e., the instance-wise probabilistic label and the group-wise probabilistic label. Compared with the existing metric-learning methods, our proposed mechanisms are capable of learning distance metrics directly from the probabilistic labels with high accuracy. We also theoretically analyze the proposed mechanisms and conduct extensive experiments on real-world datasets to verify the desirable properties of these mechanisms.

[1]  Sunita Sarawagi,et al.  Privacy-preserving Class Ratio Estimation , 2016, KDD.

[2]  Chenglin Miao,et al.  Uncorrelated Patient Similarity Learning , 2018, SDM.

[3]  S. Sathiya Keerthi,et al.  Optimization Techniques for Semi-Supervised Support Vector Machines , 2008, J. Mach. Learn. Res..

[4]  Dewei Li,et al.  Multi-view metric learning for multi-instance image classification , 2016, ArXiv.

[5]  Mengdi Huai,et al.  DIFFERENTIALLY PRIVATE SPARSE INVERSE COVARIANCE ESTIMATION , 2018, 2018 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[6]  Yong Shi,et al.  Learning from label proportions with pinball loss , 2019, Int. J. Mach. Learn. Cybern..

[7]  Tao Sun,et al.  A Probabilistic Approach for Learning with Label Proportions Applied to the US Presidential Election , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[8]  Tian Tian,et al.  Max-Margin Majority Voting for Learning from Crowds , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[10]  Lev Reyzin,et al.  On the Complexity of Learning from Label Proportions , 2017, IJCAI.

[11]  Cong Li,et al.  Reduced-Rank Local Distance Metric Learning , 2013, ECML/PKDD.

[12]  Dong Liu,et al.  SVM for Learning with Label Proportions Supplementary Material 1 , 2013 .

[13]  Zhi-Hua Zhou,et al.  Learning a distance metric from multi-instance multi-label data , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Michael I. Jordan,et al.  Distance Metric Learning with Application to Clustering with Side-Information , 2002, NIPS.

[15]  Xiaoyang Tan,et al.  Robust Distance Metric Learning in the Presence of Label Noise , 2014, AAAI.

[16]  Fei Wang,et al.  Supervised patient similarity measure of heterogeneous patient records , 2012, SKDD.

[17]  Yoram Singer,et al.  Efficient Online and Batch Learning Using Forward Backward Splitting , 2009, J. Mach. Learn. Res..

[18]  Suvrit Sra,et al.  Geometric Mean Metric Learning , 2016, ICML.

[19]  Fenglong Ma,et al.  Deep Patient Similarity Learning for Personalized Healthcare , 2018, IEEE Transactions on NanoBioscience.

[20]  Weiwei Liu,et al.  Large Margin Metric Learning for Multi-Label Prediction , 2015, AAAI.

[21]  Yong Shi,et al.  Inverse Convolutional Neural Networks for Learning from Label Proportions , 2018, 2018 IEEE/WIC/ACM International Conference on Web Intelligence (WI).

[22]  Philip S. Yu,et al.  Learning on Probabilistic Labels , 2014, SDM.

[23]  Lin Xiao,et al.  Dual Averaging Methods for Regularized Stochastic Learning and Online Optimization , 2009, J. Mach. Learn. Res..

[24]  Bernardete Ribeiro,et al.  Gaussian Process Classification and Active Learning with Multiple Annotators , 2014, ICML.

[25]  Richard Nock,et al.  (Almost) No Label No Cry , 2014, NIPS.

[26]  Joel A. Tropp,et al.  An Introduction to Matrix Concentration Inequalities , 2015, Found. Trends Mach. Learn..

[27]  Jakob Verbeek,et al.  Coordinated Local Metric Learning , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[28]  Shiyu Chang,et al.  Low-Rank Sparse Feature Selection for Patient Similarity Learning , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[29]  Shai Ben-David,et al.  Understanding Machine Learning: From Theory to Algorithms , 2014 .

[30]  Jun Huan,et al.  Sparse Compositional Local Metric Learning , 2017, KDD.

[31]  Fenglong Ma,et al.  Personalized disease prediction using a CNN-based similarity learning method , 2017, 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM).

[32]  Aidong Zhang,et al.  DTEC: Distance Transformation Based Early Time Series Classification , 2019, SDM.

[33]  Chenglin Miao,et al.  Metric Learning from Probabilistic Labels , 2018, KDD.

[34]  Stefan Rüping,et al.  SVM Classifier Estimation from Group Probabilities , 2010, ICML.

[35]  Qingyao Wu,et al.  Online Adaptive Asymmetric Active Learning for Budgeted Imbalanced Data , 2018, KDD.

[36]  Qiang Liu,et al.  Aggregating Ordinal Labels from Crowds by Minimax Conditional Entropy , 2014, ICML.

[37]  Jing Wang,et al.  Towards Mitigating the Class-Imbalance Problem for Partial Label Learning , 2018, KDD.

[38]  Chen Huang,et al.  Local Similarity-Aware Deep Feature Embedding , 2016, NIPS.

[39]  Alexandros Kalousis,et al.  Parametric Local Metric Learning for Nearest Neighbor Classification , 2012, NIPS.

[40]  Yaoliang Yu,et al.  Efficient Multiple Instance Metric Learning Using Weakly Supervised Data , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Kihyuk Sohn,et al.  Improved Deep Metric Learning with Multi-class N-pair Loss Objective , 2016, NIPS.

[42]  Inderjit S. Dhillon,et al.  Information-theoretic metric learning , 2006, ICML '07.

[43]  Cordelia Schmid,et al.  Multiple Instance Metric Learning from Automatically Labeled Bags of Faces , 2010, ECCV.

[44]  Tomer Hertz,et al.  Learning a Mahalanobis Metric from Equivalence Constraints , 2005, J. Mach. Learn. Res..

[45]  Gang Niu,et al.  Information-Theoretic Semi-Supervised Metric Learning via Entropy Regularization , 2012, Neural Computation.

[46]  Qiong Cao,et al.  Generalization bounds for metric and similarity learning , 2012, Machine Learning.

[47]  Aidong Zhang,et al.  Representation Learning for Treatment Effect Estimation from Observational Data , 2018, NeurIPS.

[48]  Byoung-Tak Zhang,et al.  Generative Local Metric Learning for Nearest Neighbor Classification , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .

[50]  Mahdieh Soleymani Baghshah,et al.  Semi-Supervised Metric Learning Using Pairwise Constraints , 2009, IJCAI.

[51]  Chenglin Miao,et al.  Deep Metric Learning: The Generalization Analysis and an Adaptive Algorithm , 2019, IJCAI.

[52]  Fenglong Ma,et al.  Multi-task Sparse Metric Learning for Monitoring Patient Similarity Progression , 2018, 2018 IEEE International Conference on Data Mining (ICDM).