Disambiguation-Free Partial Label Learning

In partial label learning, each training example is associated with a set of candidate labels among which only one is the ground-truth label. The common strategy to induce predictive model is trying to disambiguate the candidate label set, i.e., differentiating the modeling outputs of individual candidate labels. Specifically, disambiguation by differentiation can be conducted either by identifying the ground-truth label iteratively or by treating each candidate label equally. Nonetheless, the disambiguation strategy is prone to be misled by the false positive labels co-occurring with ground-truth label. In this paper, a new partial label learning strategy is studied which refrains from conducting disambiguation. Specifically, by adapting error-correcting output codes (ECOC), a simple yet effective approach named Pl-ecoc is proposed by utilizing candidate label set as an entirety . During training phase, to build binary classifier w.r.t. each column coding, any partially labeled example will be regarded as a positive or negative training example only if its candidate label set entirely falls into the coding dichotomy. During testing phase, class label for the unseen instance is determined via loss-based decoding which considers binary classifiers’ empirical performance and predictive margin. Extensive experiments show that Pl-ecoc performs favorably against state-of-the-art partial label learning approaches.

[1]  Francesco Orabona,et al.  Learning from Candidate Labeling Sets , 2010, NIPS.

[2]  Rama Chellappa,et al.  Ambiguously Labeled Learning Using Dictionaries , 2014, IEEE Transactions on Information Forensics and Security.

[3]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[4]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[5]  Jaume Amores,et al.  Multiple instance classification: Review, taxonomy and comparative study , 2013, Artif. Intell..

[6]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[7]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[8]  Rich Caruana,et al.  Classification with partial labels , 2008, KDD.

[9]  Janez Demsar,et al.  Statistical Comparisons of Classifiers over Multiple Data Sets , 2006, J. Mach. Learn. Res..

[10]  Fei Yu,et al.  Solving the Partial Label Learning Problem: An Instance-Based Approach , 2015, IJCAI.

[11]  Yoram Singer,et al.  Reducing Multiclass to Binary: A Unifying Approach for Margin Classifiers , 2000, J. Mach. Learn. Res..

[12]  Xiaojin Zhu,et al.  Introduction to Semi-Supervised Learning , 2009, Synthesis Lectures on Artificial Intelligence and Machine Learning.

[13]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[14]  Ben Taskar,et al.  Learning from ambiguously labeled images , 2009, CVPR.

[15]  Zhi-Hua Zhou,et al.  Ensemble Methods: Foundations and Algorithms , 2012 .

[16]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[17]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[18]  Sergio Escalera,et al.  On the Decoding Process in Ternary Error-Correcting Output Codes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Rong Jin,et al.  Learning with Multiple Labels , 2002, NIPS.

[20]  Xu-Ying Liu,et al.  Partial Label Learning via Feature-Aware Disambiguation , 2016, KDD.

[21]  Samuel J. Gershman,et al.  A Tutorial on Bayesian Nonparametric Models , 2011, 1106.2697.

[22]  Alexander Zien,et al.  Semi-Supervised Learning , 2006 .

[23]  Dong Xu,et al.  Learning by Associating Ambiguously Labeled Images , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Cordelia Schmid,et al.  Multiple Instance Metric Learning from Automatically Labeled Bags of Faces , 2010, ECCV.

[25]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[26]  Yoshua Bengio,et al.  Learning from Partial Labels with Minimum Entropy , 2004 .

[27]  Thierry Denoeux,et al.  Mixture Model Estimation with Soft Labels , 2008, SMPS.

[28]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[29]  Antonio Criminisi,et al.  Object categorization by learned universal visual dictionary , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[30]  Min-Ling Zhang,et al.  Confidence-Rated Discriminative Partial Label Learning , 2017, AAAI.

[31]  Patrick Vannoorenberghe,et al.  Partially Supervised Learning by a Credal EM Approach , 2005, ECSQARU.

[32]  Fei Yu,et al.  Maximum margin partial label learning , 2017, Machine Learning.

[33]  Thomas G. Dietterich,et al.  Learnability of the Superset Label Learning Problem , 2014, ICML.

[34]  Ben Taskar,et al.  Learning from Partial Labels , 2011, J. Mach. Learn. Res..

[35]  Sebastián Ventura,et al.  A Tutorial on Multilabel Learning , 2015, ACM Comput. Surv..

[36]  Sergio Escalera,et al.  An incremental node embedding technique for error correcting output codes , 2008, Pattern Recognit..

[37]  Min-Ling Zhang,et al.  Disambiguation-Free Partial Label Learning , 2017, IEEE Transactions on Knowledge and Data Engineering.

[38]  Thomas G. Dietterich,et al.  A Conditional Multinomial Mixture Model for Superset Label Learning , 2012, NIPS.

[39]  Thierry Denoeux,et al.  Learning from partially supervised data using mixture models and belief functions , 2009, Pattern Recognit..

[40]  Jesús Cid-Sueiro,et al.  Proper losses for learning from partial labels , 2012, NIPS.

[41]  L. Breiman Heuristics of instability and stabilization in model selection , 1996 .

[42]  Eyke Hüllermeier,et al.  Learning from ambiguously labeled examples , 2005, Intell. Data Anal..

[43]  Xiaoli Z. Fern,et al.  Rank-loss support instance machines for MIML instance annotation , 2012, KDD.