BigEarthNet Dataset with A New Class-Nomenclature for Remote Sensing Image Understanding

This paper presents BigEarthNet that is a large-scale Sentinel-2 multispectral image dataset with a new class nomenclature to advance deep learning (DL) studies in remote sensing (RS). BigEarthNet is made up of 590,326 image patches annotated with multi-labels provided by the CORINE Land Cover (CLC) map of 2018 based on its most thematic detailed Level-3 class nomenclature. Initial research demonstrates that some CLC classes are challenging to be accurately described by considering only Sentinel-2 images. To increase the effectiveness of BigEarthNet, in this paper we introduce an alternative class-nomenclature to allow DL models for better learning and describing the complex spatial and spectral information content of the Sentinel-2 images. This is achieved by interpreting and arranging the CLC Level-3 nomenclature based on the properties of Sentinel-2 images in a new nomenclature of 19 classes. Then, the new class-nomenclature of BigEarthNet is used within state-of-the-art DL models in the context of multi-label classification. Results show that the models trained from scratch on BigEarthNet outperform those pre-trained on ImageNet, especially in relation to some complex classes including agriculture, other vegetated and natural environments. All DL models are made publicly available at this http URL, offering an important resource to guide future progress on RS image analysis.

[1]  Artur Gil,et al.  Using a stochastic gradient boosting algorithm to analyse the effectiveness of Landsat 8 data for montado land cover mapping: Application in southern Portugal , 2016, Int. J. Appl. Earth Obs. Geoinformation.

[2]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[3]  Begüm Demir,et al.  Bigearthnet: A Large-Scale Benchmark Archive for Remote Sensing Image Understanding , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[4]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[5]  Tong Zhang,et al.  Deep Learning Based Feature Selection for Remote Sensing Scene Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[6]  Min-Ling Zhang,et al.  A Review on Multi-Label Learning Algorithms , 2014, IEEE Transactions on Knowledge and Data Engineering.

[7]  Bertrand Le Saux,et al.  Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[8]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[9]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[10]  Chih-Jen Lin,et al.  A Study on Threshold Selection for Multi-label Classification , 2007 .

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  Grigorios Tsoumakas,et al.  Mining Multi-label Data , 2010, Data Mining and Knowledge Discovery Handbook.

[13]  Xiao Xiang Zhu,et al.  Recurrently exploring class-wise attention in a hybrid convolutional and bidirectional LSTM network for multi-label aerial image classification , 2018, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[14]  Xiao Xiang Zhu,et al.  Building Instance Classification Using Street View Images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[15]  Andreas Dengel,et al.  EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Gui-Song Xia,et al.  Dirichlet-Derived Multiple Topic Scene Classification Model for High Spatial Resolution Remote Sensing Imagery , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[17]  P. Atkinson,et al.  Random Forest classification of Mediterranean land cover using multi-seasonal imagery and multi-seasonal texture , 2012 .

[18]  Gui-Song Xia,et al.  Extreme value theory-based calibration for the fusion of multiple features in high-resolution satellite scene classification , 2013 .

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Alex Krizhevsky,et al.  Learning Multiple Layers of Features from Tiny Images , 2009 .

[21]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[22]  Zhenfeng Shao,et al.  PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrieval , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[23]  Kalinka Regina Lucas Jaquie Castelo Branco,et al.  Precision Agriculture: Using Low-Cost Systems to Acquire Low-Altitude Images , 2016, IEEE Computer Graphics and Applications.

[24]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Peter Reinartz,et al.  Towards Multi-class Object Detection in Unconstrained Remote Sensing Imagery , 2018, ACCV.

[26]  Juliane Huth,et al.  The Potential of Open Geodata for Automated Large-Scale Land Use and Land Cover Classification , 2019, Remote. Sens..

[27]  Stefano Ermon,et al.  Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping , 2015, AAAI.

[28]  Ping Tang,et al.  Feature significance-based multibag-of-visual-words model for remote sensing image scene classification , 2016 .

[29]  Grigorios Tsoumakas,et al.  Multi-Label Classification: An Overview , 2007, Int. J. Data Warehous. Min..

[30]  Lorenzo Bruzzone,et al.  A Novel Approach to the Unsupervised Update of Land-Cover Maps by Classification of Time Series of Multispectral Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Haifeng Li,et al.  RSI-CB: A Large Scale Remote Sensing Image Classification Benchmark via Crowdsource Data , 2017, ArXiv.

[32]  Gerard Hazeu,et al.  European Landscape Dynamics: CORINE Land Cover Data , 2016 .

[33]  Begum Demir,et al.  A Novel Multi-Attention Driven System for Multi-Label Remote Sensing Image Classification , 2019, IGARSS 2019 - 2019 IEEE International Geoscience and Remote Sensing Symposium.

[34]  Gerard Hazeu,et al.  The EAGLE concept - A vision of a future European Land Monitoring Framework , 2013 .

[35]  Lorenzo Bruzzone,et al.  Multilabel Remote Sensing Image Retrieval Using a Semisupervised Graph-Theoretic Method , 2018, IEEE Transactions on Geoscience and Remote Sensing.