Scalable Bag of Subpaths Kernel for Learning on Hierarchical Image Representations and Multi-Source Remote Sensing Data Classification

The geographic object-based image analysis (GEOBIA) framework has gained increasing interest for the last decade. One of its key advantages is the hierarchical representation of an image, where object topological features can be extracted and modeled in the form of structured data. We thus propose to use a structured kernel relying on the concept of bag of subpaths to directly cope with such features. The kernel can be approximated using random Fourier features, allowing it to be applied on a large structure size (the number of objects in the structured data) and large volumes of data (the number of pixels or regions for training). With the so-called scalable bag of subpaths kernel (SBoSK), we also introduce a novel multi-source classification approach performing machine learning directly on a hierarchical image representation built from two images at different resolutions under the GEOBIA framework. Experiments run on an urban classification task show that the proposed approach run on a single image improves the classification overall accuracy in comparison with conventional approaches from 2% to 5% depending on the training set size and that fusing two images allows a supplementary 4% accuracy gain. Additional evaluations on public available large-scale datasets illustrate further the potential of SBoSK, with overall accuracy rates improvement ranging from 1% to 11% depending on the considered setup.

[1]  James C. Tilton,et al.  Image segmentation by region growing and spectral clustering with a natural convergence criterion , 1998, IGARSS '98. Sensing and Managing the Environment. 1998 IEEE International Geoscience and Remote Sensing. Symposium Proceedings. (Cat. No.98CH36174).

[2]  Jiali Shang,et al.  Spatial relationship-assisted classification from high-resolution remote sensing imagery , 2015, Int. J. Digit. Earth.

[3]  Gustavo Camps-Valls,et al.  Composite kernels for hyperspectral image classification , 2006, IEEE Geoscience and Remote Sensing Letters.

[4]  Jon Atli Benediktsson,et al.  A spatial-spectral kernel-based approach for the classification of remote-sensing images , 2012, Pattern Recognit..

[5]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[6]  Yingli Tian,et al.  Pyramid of Spatial Relatons for Scene-Level Land Use Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[7]  J. Aryal,et al.  A robust multi-kernel change detection framework for detecting leaf beetle defoliation using Landsat 7 ETM+ data , 2016 .

[8]  Michele Volpi,et al.  Semantic segmentation of urban scenes by learning local class interactions , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  Selim Aksoy,et al.  Image Mining Using Directional Spatial Constraints , 2010, IEEE Geoscience and Remote Sensing Letters.

[10]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[11]  Gabriele Moser,et al.  Getting pixels and regions to agree with conditional random fields , 2016, 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[12]  Jeff G. Schneider,et al.  On the Error of Random Fourier Features , 2015, UAI.

[13]  Yihong Gong,et al.  Linear spatial pyramid matching using sparse coding for image classification , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[15]  Argyros Argyridis,et al.  A Fuzzy Spatial Reasoner for Multi-Scale GEOBIA Ontologies , 2015 .

[16]  José Luis Rojo-Álvarez,et al.  Kernel-Based Framework for Multitemporal and Multisource Remote Sensing Data Classification and Change Detection , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Jing Li,et al.  Hierarchical object oriented classification using very high resolution imagery and LIDAR data over urban areas , 2009 .

[18]  Sebastian Nowozin,et al.  Structured Learning and Prediction in Computer Vision , 2011, Found. Trends Comput. Graph. Vis..

[19]  Andrea Giachetti,et al.  Scale Space Graph Representation and Kernel Matching for Non Rigid and Textured 3D Shape Retrieval , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Yannis Avrithis,et al.  To Aggregate or Not to aggregate: Selective Match Kernels for Image Search , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  Jon Atli Benediktsson,et al.  A Survey on Spectral–Spatial Classification Techniques Based on Attribute Profiles , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Ethem Alpaydin,et al.  Multiple Kernel Learning Algorithms , 2011, J. Mach. Learn. Res..

[23]  Lorenzo Bruzzone,et al.  Extended profiles with morphological attribute filters for the analysis of hyperspectral data , 2010 .

[24]  Hisashi Kashima,et al.  Fast Computation of Subpath Kernel for Trees , 2012, ICML.

[25]  Thomas Blaschke,et al.  Developing a semantic model of glacial landforms for object-based terrain classification - the example of glacial cirques , 2010 .

[26]  Florent Perronnin,et al.  Large-scale image categorization with explicit data embedding , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[28]  Germain Forestier,et al.  Knowledge-based region labeling for remote sensing image interpretation , 2012, Comput. Environ. Urban Syst..

[29]  Jean-Philippe Vert,et al.  Graph kernels based on tree patterns for molecules , 2006, Machine Learning.

[30]  K. Moffett,et al.  Remote Sens , 2015 .

[31]  Gustavo Camps-Valls,et al.  Multisource Composite Kernels for Urban-Image Classification , 2010, IEEE Geoscience and Remote Sensing Letters.

[32]  Sébastien Lefèvre,et al.  A Subpath Kernel for Learning Hierarchical Image Representations , 2015, GbRPR.

[33]  Jon Atli Benediktsson,et al.  Automatic Framework for Spectral–Spatial Classification Based on Supervised Feature Extraction and Morphological Attribute Profiles , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[34]  Yanfei Zhong,et al.  A spectral–structural bag-of-features scene classifier for very high spatial resolution remote sensing imagery , 2016 .

[35]  Zaïd Harchaoui,et al.  Image Classification with Segmentation Graph Kernels , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  AI Koan,et al.  Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning , 2008, NIPS.

[37]  Rong Jin,et al.  Nyström Method vs Random Fourier Features: A Theoretical and Empirical Comparison , 2012, NIPS.

[38]  Thomas Mensink,et al.  Improving the Fisher Kernel for Large-Scale Image Classification , 2010, ECCV.

[39]  Michael Collins,et al.  Convolution Kernels for Natural Language , 2001, NIPS.

[40]  Sébastien Lefèvre,et al.  Combining multiple resolutions into hierarchical representations for kernel-based image classification , 2016, ArXiv.

[41]  Zheng Zhang,et al.  Semisupervised Classification of Remote Sensing Images With Hierarchical Spatial Similarity , 2015, IEEE Geoscience and Remote Sensing Letters.

[42]  Nello Cristianini,et al.  Kernel Methods for Pattern Analysis , 2003, ICTAI.

[43]  Lorenzo Bruzzone,et al.  A Multilevel Context-Based System for Classification of Very High Spatial Resolution Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Curt H. Davis,et al.  A combined fuzzy pixel-based and object-based approach for classification of high-resolution multispectral data over urban areas , 2003, IEEE Trans. Geosci. Remote. Sens..

[45]  Konrad Schindler,et al.  An Overview and Comparison of Smooth Labeling Methods for Land-Cover Classification , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Nicolas Passat,et al.  Extraction of complex patterns from multiresolution remote sensing images: A hierarchical top-down methodology , 2012, Pattern Recognit..

[47]  Yuliya Tarabalka,et al.  Dynamic Ensemble Selection Approach for Hyperspectral Image Classification With Joint Spectral and Spatial Information , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[48]  Sébastien Lefèvre,et al.  Combining multiscale features for classification of hyperspectral images: A sequence-based kernel approach , 2016, 2016 8th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[49]  Jixian Zhang Multi-source remote sensing data fusion: status and trends , 2010 .

[50]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[51]  Yu Liu,et al.  A framework of region-based spatial relations for non-overlapping features and its application in object based image analysis , 2008 .

[52]  Sébastien Lefèvre,et al.  Hyperspectral image classification from multiscale description with constrained connectivity and metric learning , 2014, 2014 6th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[53]  Brian Kingsbury,et al.  How to Scale Up Kernel Methods to Be As Good As Deep Neural Nets , 2014, ArXiv.

[54]  Cristian Sminchisescu,et al.  Efficient Match Kernel between Sets of Features for Visual Recognition , 2009, NIPS.

[55]  Alexander J. Smola,et al.  Fast Kernels for String and Tree Matching , 2002, NIPS.

[56]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .

[57]  Marco Cuturi,et al.  Fast Global Alignment Kernels , 2011, ICML.

[58]  Gabriele Moser,et al.  Multimodal Classification of Remote Sensing Images: A Review and Future Directions , 2015, Proceedings of the IEEE.