A Noise-Resilient Online Learning Algorithm for Scene Classification

The proliferation of remote sensing imagery motivates a surge of research interest in image processing such as feature extraction and scene recognition, etc. Among them, scene recognition (classification) is a typical learning task that focuses on exploiting annotated images to infer the category of an unlabeled image. Existing scene classification algorithms predominantly focus on static data and are designed to learn discriminant information from clean data. They, however, suffer from two major shortcomings, i.e., the noisy label may negatively affect the learning procedure and learning from scratch may lead to a huge computational burden. Thus, they are not able to handle large-scale remote sensing images, in terms of both recognition accuracy and computational cost. To address this problem, in the paper, we propose a noise-resilient online classification algorithm, which is scalable and robust to noisy labels. Specifically, ramp loss is employed as loss function to alleviate the negative affect of noisy labels, and we iteratively optimize the decision function in Reproducing Kernel Hilbert Space under the framework of Online Gradient Descent (OGD). Experiments on both synthetic and real-world data sets demonstrate that the proposed noise-resilient online classification algorithm is more robust and sparser than state-of-the-art online classification algorithms.

[1]  Koby Crammer,et al.  On the Algorithmic Implementation of Multiclass Kernel-based Vector Machines , 2002, J. Mach. Learn. Res..

[2]  Ingo Steinwart,et al.  Sparseness of Support Vector Machines , 2003, J. Mach. Learn. Res..

[3]  Koby Crammer,et al.  Ultraconservative Online Algorithms for Multiclass Problems , 2001, J. Mach. Learn. Res..

[4]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[5]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[6]  W. Wong,et al.  On ψ-Learning , 2003 .

[7]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[8]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[9]  Peter L. Bartlett,et al.  Improved Generalization Through Explicit Optimization of Margins , 2000, Machine Learning.

[10]  Pietro Perona,et al.  Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[11]  Narasimhan Sundararajan,et al.  A Fast and Accurate Online Sequential Learning Algorithm for Feedforward Networks , 2006, IEEE Transactions on Neural Networks.

[12]  Jason Weston,et al.  Trading convexity for scalability , 2006, ICML.

[13]  Yufeng Liu,et al.  Robust Truncated Hinge Loss Support Vector Machines , 2007 .

[14]  Barbara Caputo,et al.  The projectron: a bounded kernel-based Perceptron , 2008, ICML '08.

[15]  Yoram Singer,et al.  The Forgetron: A Kernel-Based Perceptron on a Budget , 2008, SIAM J. Comput..

[16]  Ghassan Hamarneh,et al.  n -SIFT: n -Dimensional Scale Invariant Feature Transform , 2009, IEEE Trans. Image Process..

[17]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[19]  Xiaojun Yang What is Urban Remote Sensing , 2011 .

[20]  Yoram Singer,et al.  Pegasos: primal estimated sub-gradient solver for SVM , 2011, Math. Program..

[21]  Steven C. H. Hoi,et al.  Fast Bounded Online Gradient Descent Algorithms for Scalable Kernel-Based Online Learning , 2012, ICML.

[22]  Hongming Zhou,et al.  Extreme Learning Machine for Regression and Multiclass Classification , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  Shai Shalev-Shwartz,et al.  Online Learning and Online Convex Optimization , 2012, Found. Trends Mach. Learn..

[24]  Shawn D. Newsam,et al.  Geographic Image Retrieval Using Local Invariant Features , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Steven C. H. Hoi,et al.  Cost-sensitive online active learning with application to malicious URL detection , 2013, KDD.

[26]  Erchan Aptoula,et al.  Remote Sensing Image Retrieval With Global Morphological Texture Descriptors , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Bo Du,et al.  Saliency-Guided Unsupervised Feature Learning for Scene Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[29]  Charu C. Aggarwal,et al.  Data Mining: The Textbook , 2015 .

[30]  Yongjun Zhang,et al.  Content-Based High-Resolution Remote Sensing Image Retrieval via Unsupervised Feature Learning and Collaborative Affinity Metric Fusion , 2016, Remote. Sens..

[31]  Carlo Gatta,et al.  Unsupervised Deep Feature Extraction for Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Meng Joo Er,et al.  An online universal classifier for binary, multi-class and multi-label classification , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[33]  Ling Jian,et al.  Budget Online Learning Algorithm for Least Squares SVM , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[34]  Bo Li,et al.  Online User Modeling for Interactive Streaming Image Classification , 2017, MMM.

[35]  Ling Jian,et al.  A chunk updating LS-SVMs based on block Gaussian elimination method , 2017, Appl. Soft Comput..

[36]  Huan Liu,et al.  Toward online node classification on streaming networks , 2017, Data Mining and Knowledge Discovery.

[37]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[38]  A. Joshi,et al.  Information and Communication Technology for Sustainable Development , 2018 .

[39]  A. Mukherjee,et al.  Application of Remote Sensing Technology, GIS and AHP-TOPSIS Model to Quantify Urban Landscape Vulnerability to Land Use Transformation , 2018 .

[40]  A. Kafy,et al.  Integration of Remote Sensing and GIS Techniques for Flood Monitoring and Damage Assessment: A Case Study of Naogaon District, Bangladesh , 2018 .

[41]  Peng Li,et al.  Region-Wise Deep Feature Representation for Remote Sensing Images , 2018, Remote. Sens..

[42]  Ping Zhong,et al.  An Unsupervised Convolutional Feature Fusion Network for Deep Representation of Remote Sensing Images , 2018, IEEE Geoscience and Remote Sensing Letters.

[43]  Yunlong Yu,et al.  Dense Connectivity Based Two-Stream Deep Feature Fusion Framework for Aerial Scene Classification , 2018, Remote. Sens..

[44]  Siwen Bi,et al.  Development technology of principle prototype of high-resolution quantum remote sensing imaging , 2018, OPTO.

[45]  Xuelong Li,et al.  Locality and Structure Regularized Low Rank Representation for Hyperspectral Image Classification , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Ling Jian,et al.  Laplace error penalty-based M-type model detection for a class of high dimensional semiparametric models , 2019, J. Comput. Appl. Math..

[47]  Xueqi Ma,et al.  Hypergraph $p$ -Laplacian Regularization for Remotely Sensed Image Recognition , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Xuelong Li,et al.  Scene Classification With Recurrent Attention of VHR Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.