Learning User's Confidence for Active Learning

In this paper, we study the applicability of active learning (AL) in operative scenarios. More particularly, we consider the well-known contradiction between the AL heuristics, which rank the pixels according to their uncertainty, and the user's confidence in labeling, which is related to both the homogeneity of the pixel context and user's knowledge of the scene. We propose a filtering scheme based on a classifier that learns the confidence of the user in labeling, thus minimizing the queries where the user would not be able to provide a class for the pixel. The capacity of a model to learn the user's confidence is studied in detail, also showing that the effect of resolution in such a learning task. Experiments on two QuickBird images of different resolutions (with and without pansharpening) and considering committees of users prove the efficiency of the filtering scheme proposed, which maximizes the number of useful queries with respect to traditional AL.

[1]  Jacob D. Abernethy,et al.  A Collaborative Mechanism for Crowdsourcing Prediction Problems , 2011, NIPS.

[2]  Sankar K. Pal,et al.  Segmentation of multispectral remote sensing images using active support vector machines , 2004, Pattern Recognit. Lett..

[3]  Mikhail F. Kanevski,et al.  Memory-Based Cluster Sampling for Remote Sensing Image Classification , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[4]  William J. Emery,et al.  Very High Resolution Multiangle Urban Classification Analysis , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Giles M. Foody,et al.  Toward intelligent training of supervised image classifications: directing training data acquisition for SVM classification , 2004 .

[6]  Jocelyn Chanussot,et al.  Decision Fusion for the Classification of Hyperspectral Data: Outcome of the 2008 GRS-S Data Fusion Contest , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Antonio J. Plaza,et al.  Hyperspectral Image Segmentation Using a New Bayesian Approach With Active Learning , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Lorenzo Bruzzone,et al.  Batch-Mode Active-Learning Methods for the Interactive Classification of Remote Sensing Images , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[9]  David A. Cohn,et al.  Improving generalization with active learning , 1994, Machine Learning.

[10]  Melba M. Crawford,et al.  View Generation for Multiview Maximum Disagreement Based Active Learning for Hyperspectral Image Classification , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Jon Atli Benediktsson,et al.  Morphological Attribute Profiles for the Analysis of Very High Resolution Images , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Antonio J. Plaza,et al.  Semisupervised Hyperspectral Image Segmentation Using Multinomial Logistic Regression With Active Learning , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Daniel H. Grollman,et al.  Dogged Learning for Robots , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[14]  Mikhail F. Kanevski,et al.  A Survey of Active Learning Algorithms for Supervised Remote Sensing Image Classification , 2011, IEEE Journal of Selected Topics in Signal Processing.

[15]  Maja J. Matarić,et al.  A framework for learning from demonstration, generalization and practice in human-robot domains , 2003 .

[16]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[17]  Lorenzo Bruzzone,et al.  A Fast Cluster-Assumption Based Active-Learning Technique for Classification of Remote Sensing Images , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Tetsunari Inamura Masayuki Inaba Hirochika.  Acquisition of Probabilistic Behavior Decision Model based on the Interactive Teaching Method , 2001 .

[19]  Julien Radoux,et al.  Bayesian Data Fusion for Adaptable Image Pansharpening , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[20]  T. Esch,et al.  Monitoring urbanization in mega cities from space , 2012 .

[21]  William J. Emery,et al.  Active Learning Methods for Remote Sensing Image Classification , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[23]  Pierre Soille,et al.  Morphological Image Analysis , 1999 .

[24]  Devis Tuia,et al.  Remote Sensing Image Processing. Synthesis Lectures on Image, Video, and Multimedia Processing. , 2011 .

[25]  M. S. Moran,et al.  Remote Sensing for Crop Management , 2003 .

[26]  Luis Gómez-Chova,et al.  Remote Sensing Image Processing , 2011, Remote Sensing Image Processing.

[27]  Samy Bengio,et al.  Torch: a modular machine learning software library , 2002 .

[28]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[29]  William J. Emery,et al.  Using active learning to adapt remote sensing image classifiers , 2011 .

[30]  Michael E. Schaepman,et al.  A review on reflective remote sensing and data assimilation techniques for enhanced agroecosystem modeling , 2007, Int. J. Appl. Earth Obs. Geoinformation.

[31]  Gustavo Camps-Valls,et al.  Semisupervised Classification of Remote Sensing Images With Active Queries , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Pietro Perona,et al.  Crowdclustering , 2011, NIPS.

[33]  Lorenzo Bruzzone,et al.  A novel active learning strategy for domain adaptation in the classification of remote sensing images , 2011, 2011 IEEE International Geoscience and Remote Sensing Symposium.

[34]  Manuela M. Veloso,et al.  Interactive Policy Learning through Confidence-Based Autonomy , 2014, J. Artif. Intell. Res..

[35]  Giles M. Foody,et al.  The use of small training sets containing mixed pixels for accurate hard image classification: Training on mixed spectral responses for classification by a SVM , 2006 .

[36]  Melba M. Crawford,et al.  Active Learning via Multi-View and Local Proximity Co-Regularization for Hyperspectral Image Classification , 2011, IEEE Journal of Selected Topics in Signal Processing.

[37]  Alan Fern,et al.  Active Imitation Learning via State Queries , 2011 .

[38]  Farid Melgani,et al.  Support Vector Machine Active Learning Through Significance Space Construction , 2011, IEEE Geoscience and Remote Sensing Letters.

[39]  Gustavo Camps-Valls,et al.  Remote sensing image segmentation by active queries , 2012, Pattern Recognit..

[40]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.