Learning from Label Proportions: A Mutual Contamination Framework

Learning from label proportions (LLP) is a weakly supervised setting for classification in which unlabeled training instances are grouped into bags, and each bag is annotated with the proportion of each class occurring in that bag. Prior work on LLP has yet to establish a consistent learning procedure, nor does there exist a theoretically justified, general purpose training criterion. In this work we address these two issues by posing LLP in terms of mutual contamination models (MCMs), which have recently been applied successfully to study various other weak supervision settings. In the process, we establish several novel technical results for MCMs, including unbiased losses and generalization error bounds under non-iid sampling plans. We also point out the limitations of a common experimental setting for LLP, and propose a new one based on our MCM framework.

[1]  Gilles Blanchard,et al.  Decontamination of Mutually Contaminated Models , 2014, AISTATS.

[2]  L. Dery,et al.  Weakly supervised classification in high energy physics , 2017, Journal of High Energy Physics.

[3]  M. Talagrand,et al.  Probability in Banach Spaces: Isoperimetry and Processes , 1991 .

[4]  Nando de Freitas,et al.  Learning about Individuals from Group Statistics , 2005, UAI.

[5]  Masashi Sugiyama,et al.  On Symmetric Losses for Learning from Corrupted Labels , 2019, ICML.

[6]  Bo Wang,et al.  Linear Twin SVM for Learning from Label Proportions , 2015, 2015 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT).

[7]  Gilles Blanchard,et al.  Classification with Asymmetric Label Noise: Consistency and Maximal Denoising , 2013, COLT.

[8]  Jianxin Zhang,et al.  Learning from Multiple Corrupted Sources, with Application to Learning from Label Proportions , 2019, ArXiv.

[9]  Bo Wang,et al.  Learning with label proportions based on nonparallel support vector machines , 2017, Knowl. Based Syst..

[10]  Michael I. Jordan,et al.  Convexity, Classification, and Risk Bounds , 2006 .

[11]  Gilles Blanchard,et al.  Decontamination of Mutual Contamination Models , 2017, J. Mach. Learn. Res..

[12]  Cheng Soon Ong,et al.  Learning from Corrupted Binary Labels via Class-Probability Estimation , 2015, ICML.

[13]  Philip D. Plowright,et al.  Convexity , 2019, Optimization for Chemical and Biochemical Engineering.

[14]  Aravind Srinivasan,et al.  Randomized Distributed Edge Coloring via an Extension of the Chernoff-Hoeffding Bounds , 1997, SIAM J. Comput..

[15]  Yong Shi,et al.  Learning from Label Proportions with Generative Adversarial Networks , 2019, NeurIPS.

[16]  Jack Edmonds,et al.  Maximum matching and a polyhedron with 0,1-vertices , 1965 .

[17]  Aron Culotta,et al.  Co-Training for Demographic Classification Using Deep Learning from Label Proportions , 2017, 2017 IEEE International Conference on Data Mining Workshops (ICDMW).

[18]  Nagarajan Natarajan,et al.  Cost-Sensitive Learning with Noisy Labels , 2017, J. Mach. Learn. Res..

[19]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[20]  Iñaki Inza,et al.  Learning Bayesian network classifiers from label proportions , 2013, Pattern Recognit..

[21]  Iñaki Inza,et al.  Fitting the data from embryo implantation prediction: Learning from label proportions , 2018, Statistical methods in medical research.

[22]  Yong Shi,et al.  Learning from label proportions with pinball loss , 2019, Int. J. Mach. Learn. Cybern..

[23]  Tao Sun,et al.  A Probabilistic Approach for Learning with Label Proportions Applied to the US Presidential Election , 2017, 2017 IEEE International Conference on Data Mining (ICDM).

[24]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[25]  Stefan Rüping,et al.  SVM Classifier Estimation from Group Probabilities , 2010, ICML.

[26]  Alexander J. Smola,et al.  Estimating labels from label proportions , 2008, ICML '08.

[27]  ShiYong,et al.  Learning with label proportions based on nonparallel support vector machines , 2017 .

[28]  Russell Impagliazzo,et al.  Constructive Proofs of Concentration Bounds , 2010, APPROX-RANDOM.

[29]  Katharina Morik,et al.  Learning from Label Proportions by Optimizing Cluster Model Selection , 2011, ECML/PKDD.

[30]  Dong Liu,et al.  $\propto$SVM for learning with label proportions , 2013, ICML 2013.

[31]  Shih-Fu Chang,et al.  On Learning with Label Proportions , 2014, ArXiv.

[32]  Bo Wang,et al.  Learning from label proportions on high-dimensional data , 2018, Neural Networks.

[33]  Niall Twomey,et al.  LABEL PROPAGATION FOR LEARNING WITH LABEL PROPORTIONS , 2018, 2018 IEEE 28th International Workshop on Machine Learning for Signal Processing (MLSP).

[34]  Richard Nock,et al.  (Almost) No Label No Cry , 2014, NIPS.

[35]  Marleen de Bruijne,et al.  Deep Learning from Label Proportions for Emphysema Quantification , 2018, MICCAI.

[36]  Lucas Beyer,et al.  Deep multi-class learning from label proportions , 2019, ArXiv.

[37]  Wenxian Yu,et al.  Learning from label proportions for SAR image classification , 2017, EURASIP J. Adv. Signal Process..

[38]  Aditya Krishna Menon,et al.  An Average Classification Algorithm , 2015, ArXiv.

[39]  Tao Chen,et al.  Object-Based Visual Sentiment Concept Analysis and Application , 2014, ACM Multimedia.

[40]  Ming-Syan Chen,et al.  Video Event Detection by Inferring Temporal Instance Labels , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Andreas Christmann,et al.  Support vector machines , 2008, Data Mining and Knowledge Discovery Handbook.

[42]  Bin Liu,et al.  Kernel K-means Based Framework for Aggregate Outputs Classification , 2009, 2009 IEEE International Conference on Data Mining Workshops.

[43]  Ron Meir,et al.  Generalization Error Bounds for Bayesian Mixture Algorithms , 2003, J. Mach. Learn. Res..

[44]  Fan Li,et al.  Alter-CNN: An Approach to Learning from Label Proportions with Application to Ice-Water Classification , 2015 .

[45]  Hsuan-Tien Lin,et al.  Learning from Label Proportions with Consistency Regularization , 2019, ACML.

[46]  Zhiquan Qi,et al.  Learning With Label Proportions via NPSVM , 2017, IEEE Transactions on Cybernetics.

[47]  Colin McDiarmid,et al.  Surveys in Combinatorics, 1989: On the method of bounded differences , 1989 .