Very High Resolution Remote Sensing Imagery Classification Using a Fusion of Random Forest and Deep Learning Technique—Subtropical Area for Example

Recently, convolutional neural networks (CNNs) showed excellent performance in many tasks, such as computer vision and remote sensing semantic segmentation. Especially, the ability to learn high-representation features of CNN draws much attention. And random forest (RF) algorithm, on the other hand, is widely applied for variables selection, classification, and regression. Based on the previous fusion models that fused CNN with the other models, such as conditional random fields (CRFs), support vector machine (SVM), and RF, this article tested a method based on the fusion of an RF classifier and the CNN for a very high resolution remote sensing (VHRRS) based forests mapping. The study area is located in the south of China and the main purpose was to precisely distinguish Lei bamboo forests from the other subtropical forests. The main novelties of this article are as follows. First, a test was conducted to confirm if a fusion of CNN and RF make an improvement in the VHRRS information extraction. Second, based on RF, variables with high importance were selected. Then, a test was again conducted to confirm if the learning from the selected variables will further give better results.

[1]  Nataliia Kussul,et al.  Deep Learning Classification of Land Cover and Crop Types Using Remote Sensing Data , 2017, IEEE Geoscience and Remote Sensing Letters.

[2]  Bertrand Le Saux,et al.  Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[3]  Haokui Zhang,et al.  Deep learning for remote sensing image classification: A survey , 2018, WIREs Data Mining Knowl. Discov..

[4]  Shilei Sun,et al.  Vehicle detection from high-resolution aerial images using spatial pyramid pooling-based deep convolutional neural networks , 2016, Multimedia Tools and Applications.

[5]  Jianya Gong,et al.  Monitoring ecosystem service change in the City of Shenzhen by the use of high‐resolution remotely sensed imagery and deep learning , 2019, Land Degradation & Development.

[6]  Achim Zeileis,et al.  Bias in random forest variable importance measures: Illustrations, sources and a solution , 2007, BMC Bioinformatics.

[7]  Mariana Belgiu,et al.  Random forest in remote sensing: A review of applications and future directions , 2016 .

[8]  Haifeng Li,et al.  RSI-CB: A Large Scale Remote Sensing Image Classification Benchmark via Crowdsource Data , 2017, ArXiv.

[9]  Arko Lucieer,et al.  Object-based random forest classification of Landsat ETM+ and WorldView-2 satellite imagery for mapping lowland native grassland communities in Tasmania, Australia , 2018, Int. J. Appl. Earth Obs. Geoinformation.

[10]  Lorenzo Bruzzone,et al.  A Novel Transductive SVM for Semisupervised Classification of Remote-Sensing Images , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Yi Li,et al.  Deformable Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Chunxiang Cao,et al.  Random Forest Variable Importance Spectral Indices Scheme for Burnt Forest Recovery Monitoring - Multilevel RF-VIMP , 2018, Remote. Sens..

[13]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Peerapon Vateekul,et al.  Road Segmentation of Remotely-Sensed Images Using Deep Convolutional Neural Networks with Landscape Metrics and Conditional Random Fields , 2017, Remote. Sens..

[15]  Jianping Shi,et al.  Distinguishing Cloud and Snow in Satellite Images via Deep Convolutional Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[16]  Ke Wang,et al.  Integration of texture and landscape features into object-based classification for delineating Torreya using IKONOS imagery , 2012 .

[17]  Andrew Crooks,et al.  A Critical Review of High and Very High-Resolution Remote Sensing Approaches for Detecting and Mapping Slums: Trends, Challenges and Emerging Opportunities , 2018 .

[18]  Xueliang Zhang,et al.  Deep learning in remote sensing applications: A meta-analysis and review , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[19]  Miaozhong Xu,et al.  DenseNet-Based Depth-Width Double Reinforced Deep Learning Neural Network for High-Resolution Remote Sensing Image Per-Pixel Classification , 2018, Remote. Sens..

[20]  Tim Appelhans,et al.  Improving the accuracy of rainfall rates from optical satellite sensors with machine learning — A random forests-based approach applied to MSG SEVIRI , 2014 .

[21]  Konstantinos Karantzalos,et al.  BENCHMARKING DEEP LEARNING FRAMEWORKS FOR THE CLASSIFICATION OF VERY HIGH RESOLUTION SATELLITE MULTISPECTRAL DATA , 2016 .

[22]  Yuichiro Hayashi,et al.  An application of cascaded 3D fully convolutional networks for medical image segmentation , 2018, Comput. Medical Imaging Graph..

[23]  Peng Li,et al.  Region-Wise Deep Feature Representation for Remote Sensing Images , 2018, Remote. Sens..

[24]  P. Hostert,et al.  Forest disturbances, forest recovery, and changes in forest types across the Carpathian ecoregion from 1985 to 2010 based on Landsat image composites , 2014 .

[25]  Alexis J. Comber,et al.  Random forest classification of salt marsh vegetation habitats using quad-polarimetric airborne SAR, elevation and optical RS data , 2014 .

[26]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[27]  Xi Chen,et al.  Hand pose estimation in depth image using CNN and random forest , 2018, International Symposium on Multispectral Image Processing and Pattern Recognition.

[28]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[29]  Jinwei Dong,et al.  Mapping paddy rice planting area in northeastern Asia with Landsat 8 images, phenology-based algorithm and Google Earth Engine. , 2016, Remote sensing of environment.

[30]  Weiming Shen,et al.  Bearing Fault Diagnosis Method Based on Deep Convolutional Neural Network and Random Forest Ensemble Learning , 2019, Sensors.

[31]  Ulrich Michel,et al.  Decision fusion and non-parametric classifiers for land use mapping using multi-temporal RapidEye data , 2015 .

[32]  Guillermo Sapiro,et al.  ForestHash: Semantic Hashing With Shallow Random Forests and Tiny Convolutional Networks , 2017, ECCV.

[33]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[34]  Timothy A. Warner,et al.  Implementation of machine-learning classification in remote sensing: an applied review , 2018 .

[35]  Shuyuan Yang,et al.  Deep Fully Convolutional Network-Based Spatial Distribution Prediction for Hyperspectral Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[36]  Johannes R. Sveinsson,et al.  Classification of hyperspectral data from urban areas based on extended morphological profiles , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[37]  M.,et al.  Statistical and Structural Approaches to Texture , 2022 .

[38]  John R. Miller,et al.  Hyperspectral vegetation indices and novel algorithms for predicting green LAI of crop canopies: Modeling and validation in the context of precision agriculture , 2004 .

[39]  Jamie Sherrah,et al.  Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagery , 2016, ArXiv.

[40]  Eiji Hirata,et al.  Decomposition patterns of leaf litter of seven common canopy species in a subtropical forest: N and P dynamics , 2005, Plant and Soil.

[41]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[42]  Shaowen Wang,et al.  A high-performance and in-season classification system of field-level crop types using time-series Landsat data and a machine learning approach , 2018, Remote Sensing of Environment.

[43]  Claudia Notarnicola,et al.  Review of Machine Learning Approaches for Biomass and Soil Moisture Retrievals from Remote Sensing Data , 2015, Remote. Sens..

[44]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Caiyun Zhang,et al.  Combining object-based texture measures with a neural network for vegetation mapping in the Everglades from hyperspectral imagery , 2012 .

[46]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[47]  Ling Tong,et al.  Rice biomass retrieval from multitemporal ground-based scatterometer data and RADARSAT-2 images using neural networks , 2013 .

[48]  Gang Fu,et al.  Classification for High Resolution Remote Sensing Imagery Using a Fully Convolutional Network , 2017, Remote. Sens..

[49]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[50]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[51]  Weifeng Li,et al.  Comparing Machine Learning Classifiers for Object-Based Land Cover Classification Using Very High Resolution Imagery , 2014, Remote. Sens..

[52]  Martin Kappas,et al.  Comparison of Random Forest, k-Nearest Neighbor, and Support Vector Machine Classifiers for Land Cover Classification Using Sentinel-2 Imagery , 2017, Sensors.

[53]  Bisheng Yang,et al.  Computing multiple aggregation levels and contextual features for road facilities recognition using mobile laser scanning data , 2017 .

[54]  Kellie J. Archer,et al.  Empirical characterization of random forest variable importance measures , 2008, Comput. Stat. Data Anal..

[55]  Giuseppe Scarpa,et al.  A CNN-Based Fusion Method for Feature Extraction from Sentinel Data , 2018, Remote. Sens..

[56]  Fang Liu,et al.  A Novel Neural Network for Remote Sensing Image Matching , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[57]  David A Clausiyz,et al.  Towards a Novel Approach for Texture Segmentation of Sar Sea Ice Imagery , 2022 .

[58]  Xin Pan,et al.  A hybrid MLP-CNN classifier for very fine resolution remotely sensed image classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[59]  Xin Pan,et al.  An object-based convolutional neural network (OCNN) for urban land use classification , 2018, Remote Sensing of Environment.

[60]  Enhua Wu,et al.  Squeeze-and-Excitation Networks , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[61]  Hassan Ghassemian,et al.  A review of remote sensing image fusion methods , 2016, Inf. Fusion.

[62]  S. Shataee,et al.  Forest Attributes Estimation Using Aerial Laser Scanner and TM Data , 2013 .

[63]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[64]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  J. Macgregor,et al.  Image texture analysis: methods and comparisons , 2004 .

[66]  Gérard Dedieu,et al.  Assessing the robustness of Random Forests to map land cover with high resolution satellite image time series over large areas , 2016 .

[67]  Yihua Tan,et al.  Automatic Extraction of Built-Up Areas From Panchromatic and Multispectral Remote Sensing Images Using Double-Stream Deep Convolutional Neural Networks , 2018, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[68]  Tao Li,et al.  DCNR: deep cube CNN with random forest for hyperspectral image classification , 2018, Multimedia Tools and Applications.

[69]  Zhe Zhu,et al.  Automated cloud, cloud shadow, and snow detection in multitemporal Landsat data: An algorithm designed specifically for monitoring land cover change , 2014 .

[70]  Yoshua Bengio,et al.  Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanism , 2016, NAACL.

[71]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[72]  Zhanyi Hu,et al.  Semantic labeling of high-resolution aerial images using an ensemble of fully convolutional networks , 2017 .