Memory-Based Cluster Sampling for Remote Sensing Image Classification

In this paper, we address the problem of semi-automatic definition of training sets for the classification of remotely sensed images. We propose two approaches based on active learning aiming at removing both the proximal (low diversity) and the dense (low exploration during iterations) sampling redundancies. The first is encountered when several samples carrying similar spectral information are selected by the algorithm, while the second occurs when the heuristic is unable to explore undiscovered parts of the feature space during iterations. For this purpose, kernel k-means is used to cluster a set of uncertain candidates in the same space spanned by the kernel function defined in the SVM classification step. Two heuristics are proposed to maximize the speed of convergence to high classification accuracies: The first is based on binary hierarchical partitioning of the set of selected uncertain samples, while the second extends this approach by considering memory in the selection and thus dynamically adapts to the problem throughout the iterations. Experiments on both VHR and hyperspectral imagery confirm fast convergence of the algorithm, that outperforms state-of-the-art sampling schemes.

[1]  Johannes R. Sveinsson,et al.  Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles , 2008, 2007 IEEE International Geoscience and Remote Sensing Symposium.

[2]  Andreas Vlachos,et al.  A stopping criterion for active learning , 2008, Computer Speech and Language.

[3]  William J. Emery,et al.  Active Learning Methods for Remote Sensing Image Classification , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[4]  David D. Lewis,et al.  Heterogeneous Uncertainty Sampling for Supervised Learning , 1994, ICML.

[5]  Greg Schohn,et al.  Less is More: Active Learning with Support Vector Machines , 2000, ICML.

[6]  Sankar K. Pal,et al.  Segmentation of multispectral remote sensing images using active support vector machines , 2004, Pattern Recognit. Lett..

[7]  Lawrence Carin,et al.  Detection of Unexploded Ordnance via Efficient Semisupervised and Active Learning , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Alexander J. Smola,et al.  Learning with kernels , 1998 .

[9]  J. Mercer.  Functions of positive and negative type, and their connection with the theory of integral equations , 1909 .

[10]  Xiaowei Xu,et al.  Representative Sampling for Text Classification Using Support Vector Machines , 2003, ECIR.

[11]  Francesca Bovolo,et al.  A Novel Approach to Unsupervised Change Detection Based on a Semisupervised SVM and a Similarity Measure , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Joydeep Ghosh,et al.  Investigation of the random forest framework for classification of hyperspectral data , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[13]  Lorenzo Bruzzone,et al.  Kernel methods for remote sensing data analysis , 2009 .

[14]  Sebastian Thrun,et al.  Text Classification from Labeled and Unlabeled Documents using EM , 2000, Machine Learning.

[15]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Lorenzo Bruzzone,et al.  Classification of hyperspectral remote sensing images with support vector machines , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Samy Bengio,et al.  Torch: a modular machine learning software library , 2002 .

[18]  Alexander Y. Liu,et al.  Active Learning with Spatially Sensitive Labeling Costs , 2008 .

[19]  Mikhail F. Kanevski,et al.  A Survey of Active Learning Algorithms for Supervised Remote Sensing Image Classification , 2011, IEEE Journal of Selected Topics in Signal Processing.

[20]  Raymond J. Mooney,et al.  Diverse ensembles for active learning , 2004, ICML.

[21]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[22]  Lorenzo Bruzzone,et al.  Kernel-based methods for hyperspectral image classification , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[23]  Lawrence O. Hall,et al.  Active learning to recognize multiple types of plankton , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[24]  Klaus Brinker,et al.  Incorporating Diversity in Active Learning with Support Vector Machines , 2003, ICML.

[25]  G. Foody.  Thematic map comparison: Evaluating the statistical significance of differences in classification accuracy , 2004 .

[26]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[27]  Giles M. Foody,et al.  Toward intelligent training of supervised image classifications: directing training data acquisition for SVM classification , 2004 .

[28]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2004 .

[29]  Goo Jun,et al.  An Efficient Active Learning Algorithm with Knowledge Transfer for Hyperspectral Data Analysis , 2008, IGARSS 2008 - 2008 IEEE International Geoscience and Remote Sensing Symposium.

[30]  Giles M. Foody,et al.  Training set size requirements for the classification of a specific class , 2006 .

[31]  Francesca Bovolo,et al.  Supervised change detection in VHR images using contextual information and support vector machines , 2013, Int. J. Appl. Earth Obs. Geoinformation.

[32]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[33]  Lawrence Carin,et al.  Detection of buried targets via active selection of labeled data: application to sensing subsurface UXO , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[35]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[36]  Naoki Abe,et al.  Query Learning Strategies Using Boosting and Bagging , 1998, ICML.

[37]  Burr Settles,et al.  Active Learning Literature Survey , 2009 .

[38]  David J. C. MacKay,et al.  Information-Based Objective Functions for Active Data Selection , 1992, Neural Computation.

[39]  Nello Cristianini,et al.  Query Learning with Large Margin Classi ersColin , 2000 .

[40]  Rujie Liu,et al.  SVM-based active feedback in image retrieval using clustering and unlabeled data , 2008, Pattern Recognit..

[41]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[42]  Mark A. Girolami,et al.  Mercer kernel-based clustering in feature space , 2002, IEEE Trans. Neural Networks.

[43]  Gustavo Camps-Valls,et al.  Multisource Composite Kernels for Urban-Image Classification , 2010, IEEE Geoscience and Remote Sensing Letters.

[44]  Shlomo Argamon,et al.  Committee-Based Sampling For Training Probabilistic Classi(cid:12)ers , 1995 .

[45]  H. Sebastian Seung,et al.  Query by committee , 1992, COLT '92.

[46]  Lorenzo Bruzzone,et al.  Batch-Mode Active-Learning Methods for the Interactive Classification of Remote Sensing Images , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[47]  I. Dhillon,et al.  A Unified View of Kernel k-means , Spectral Clustering and Graph Cuts , 2004 .

[48]  Luis Alonso,et al.  Robust support vector method for hyperspectral data classification and knowledge discovery , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[49]  H. Sebastian Seung,et al.  Selective Sampling Using the Query by Committee Algorithm , 1997, Machine Learning.

[50]  Joydeep Ghosh,et al.  An Active Learning Approach to Hyperspectral Data Classification , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[51]  Andrew McCallum,et al.  Toward Optimal Active Learning through Sampling Estimation of Error Reduction , 2001, ICML.