Active Learning With Optimal Instance Subset Selection

Active learning (AL) traditionally relies on some instance-based utility measures (such as uncertainty) to assess individual instances and label the ones with the maximum values for training. In this paper, we argue that such approaches cannot produce good labeling subsets mainly because instances are evaluated independently without considering their interactions, and individuals with maximal ability do not necessarily form an optimal instance subset for learning. Alternatively, we propose to achieve AL with optimal subset selection (ALOSS), where the key is to find an instance subset with a maximum utility value. To achieve the goal, ALOSS simultaneously considers the following: 1) the importance of individual instances and 2) the disparity between instances, to build an instance-correlation matrix. As a result, AL is transformed to a semidefinite programming problem to select a k-instance subset with a maximum utility value. Experimental results demonstrate that ALOSS outperforms state-of-the-art approaches for AL.

[1]  Dale Schuurmans,et al.  Discriminative Batch Mode Active Learning , 2007, NIPS.

[2]  Andrew McCallum,et al.  Employing EM and Pool-Based Active Learning for Text Classification , 1998, ICML.

[3]  Wei-Yun Yau,et al.  Active Learning for Solving the Incomplete Data Problem in Facial Age Classification by the Furthest Nearest-Neighbor Criterion , 2011, IEEE Transactions on Image Processing.

[4]  John Langford,et al.  Importance weighted active learning , 2008, ICML '09.

[5]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[6]  Yi Zhang,et al.  Incorporating Diversity and Density in Active Learning for Relevance Feedback , 2007, ECIR.

[7]  Daphne Koller,et al.  Support Vector Machine Active Learning with Applications to Text Classification , 2000, J. Mach. Learn. Res..

[8]  Fei Wang,et al.  Two Heads Better Than One: Metric+Active Learning and its Applications for IT Service Classification , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[9]  David P. Williamson,et al.  Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming , 1995, JACM.

[10]  Arnold W. M. Smeulders,et al.  Active learning using pre-clustering , 2004, ICML.

[11]  Naoki Abe,et al.  Query Learning Strategies Using Boosting and Bagging , 1998, ICML.

[12]  Xiaodong Lin,et al.  Active Learning From Stream Data Using Optimal Weight Classifier Ensemble , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[14]  Andrew McCallum,et al.  Toward Optimal Active Learning through Sampling Estimation of Error Reduction , 2001, ICML.

[15]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[16]  Andrew McCallum,et al.  Reducing Labeling Effort for Structured Prediction Tasks , 2005, AAAI.

[17]  Masakazu Kojima,et al.  Implementation and evaluation of SDPA 6.0 (Semidefinite Programming Algorithm 6.0) , 2003, Optim. Methods Softw..

[18]  Jaime G. Carbonell,et al.  Active machine learning for transmembrane helix prediction , 2010, BMC Bioinformatics.

[19]  Bin Li,et al.  A survey on instance selection for active learning , 2012, Knowledge and Information Systems.

[20]  Weiru Liu,et al.  Under Consideration for Publication in Knowledge and Information Systems a General Framework for Measuring Inconsistency through Minimal Inconsistent Sets , 2022 .

[21]  Longin Jan Latecki,et al.  Improving SVM classification on imbalanced time series data sets with ghost points , 2011, Knowledge and Information Systems.

[22]  Daphne Koller,et al.  Support Vector Machine Active Learning with Application sto Text Classification , 2000, ICML.

[23]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[24]  Rong Jin,et al.  Batch mode active learning and its application to medical image classification , 2006, ICML.

[25]  Jaime G. Carbonell,et al.  Proactive learning: cost-sensitive active learning with multiple imperfect oracles , 2008, CIKM '08.

[26]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[27]  William Nick Street,et al.  Ensemble Pruning Via Semi-definite Programming , 2006, J. Mach. Learn. Res..

[28]  María José del Jesús,et al.  An overview on subgroup discovery: foundations and applications , 2011, Knowledge and Information Systems.

[29]  Jennifer G. Dy,et al.  Active Learning from Crowds , 2011, ICML.

[30]  Klaus Brinker,et al.  Incorporating Diversity in Active Learning with Support Vector Machines , 2003, ICML.

[31]  Russell Greiner,et al.  Optimistic Active-Learning Using Mutual Information , 2007, IJCAI.

[32]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[33]  Mark Craven,et al.  An Analysis of Active Learning Strategies for Sequence Labeling Tasks , 2008, EMNLP.

[34]  Lise Getoor,et al.  Active Learning for Networked Data , 2010, ICML.