Ambiguity-Based Multiclass Active Learning

Most existing works on active learning (AL) focus on binary classification problems, which limit their applications in various real-world scenarios. One solution to multiclass AL (MAL) is evaluating the informativeness of unlabeled samples by an uncertainty model and selecting the most uncertain one for query. In this paper, an ambiguity-based strategy is proposed to tackle this problem by applying a possibility approach. First, the possibilistic memberships of unlabeled samples in the multiple classes are calculated from the one-against-all-based support vector machine model. Then, by employing fuzzy logic operators, these memberships are aggregated into a new concept named k-order ambiguity, which estimates the risk of labeling a sample among k classes. Afterward, the k-order ambiguities are used to form an overall ambiguity measure to evaluate the uncertainty of the unlabeled samples. Finally, the sample with the maximum ambiguity is selected for query, and a new MAL strategy is developed. Experiments demonstrate the feasibility and effectiveness of the proposed method.

[1]  L. Mascarilla,et al.  A third way to design pattern classifiers with reject options , 2002, 2002 Annual Meeting of the North American Fuzzy Information Processing Society Proceedings. NAFIPS-FLINT 2002 (Cat. No. 02TH8622).

[2]  Ishwar K. Sethi,et al.  Confidence-based active learning , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Tao Xiang,et al.  Finding Rare Classes: Active Learning with Generative and Discriminative Models , 2013, IEEE Transactions on Knowledge and Data Engineering.

[4]  Nikolaos Papanikolopoulos,et al.  Scalable Active Learning for Multiclass Image Classification , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Daphne Koller,et al.  Active learning: theory and applications , 2001 .

[6]  Rong Jin,et al.  Batch mode active learning and its application to medical image classification , 2006, ICML.

[7]  Sam Kwong,et al.  Inconsistency-based active learning for support vector machines , 2012, Pattern Recognit..

[8]  D. Dubois,et al.  On Possibility/Probability Transformations , 1993 .

[9]  George J. Klir,et al.  Fuzzy sets and fuzzy logic - theory and applications , 1995 .

[10]  Yu-Lin He,et al.  Learning ELM-Tree from big data based on uncertainty reduction , 2015, Fuzzy Sets Syst..

[11]  Xizhao Wang,et al.  Maximum Ambiguity-Based Sample Selection in Fuzzy Decision Tree Induction , 2012, IEEE Transactions on Knowledge and Data Engineering.

[12]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[13]  Ryan M. Rifkin,et al.  In Defense of One-Vs-All Classification , 2004, J. Mach. Learn. Res..

[14]  Rong Yan,et al.  Automatically labeling video data using multi-class active learning , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[15]  Rong Yan,et al.  Multi-class active learning for video semantic feature extraction , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[16]  Chih-Jen Lin,et al.  Probability Estimates for Multi-class Classification by Pairwise Coupling , 2003, J. Mach. Learn. Res..

[17]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[18]  L. Bottou,et al.  1 Support Vector Machine Solvers , 2007 .

[19]  Laurent Mascarilla,et al.  A k-order fuzzy OR operator for pattern classification with k-order ambiguity rejection , 2008, Fuzzy Sets Syst..

[20]  Sam Kwong,et al.  Fuzzy-Rough-Set-Based Active Learning , 2014, IEEE Transactions on Fuzzy Systems.

[21]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.