Review and Evaluation of Deep Learning Architectures for Efficient Land Cover Mapping with UAS Hyper-Spatial Imagery: A Case Study Over a Wetland

Deep learning has already been proved as a powerful state-of-the-art technique for many image understanding tasks in computer vision and other applications including remote sensing (RS) image analysis. Unmanned aircraft systems (UASs) offer a viable and economical alternative to a conventional sensor and platform for acquiring high spatial and high temporal resolution data with high operational flexibility. Coastal wetlands are among some of the most challenging and complex ecosystems for land cover prediction and mapping tasks because land cover targets often show high intra-class and low inter-class variances. In recent years, several deep convolutional neural network (CNN) architectures have been proposed for pixel-wise image labeling, commonly called semantic image segmentation. In this paper, some of the more recent deep CNN architectures proposed for semantic image segmentation are reviewed, and each model’s training efficiency and classification performance are evaluated by training it on a limited labeled image set. Training samples are provided using the hyper-spatial resolution UAS imagery over a wetland area and the required ground truth images are prepared by manual image labeling. Experimental results demonstrate that deep CNNs have a great potential for accurate land cover prediction task using UAS hyper-spatial resolution images. Some simple deep learning architectures perform comparable or even better than complex and very deep architectures with remarkably fewer training epochs. This performance is especially valuable when limited training samples are available, which is a common case in most RS applications.

[1]  Gang Wang,et al.  Deep Learning-Based Classification of Hyperspectral Data , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[3]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Farid Melgani,et al.  Nearest Neighbor Classification of Remote Sensing Images With the Maximal Margin Principle , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Chuyen Nguyen,et al.  Unsupervised Clustering Method for Complexity Reduction of Terrestrial Lidar Data in Marshes , 2018, Remote. Sens..

[6]  P. Gong,et al.  Object-based analysis and change detection of major wetland cover types and their classification uncertainty during the low water period at Poyang Lake, China , 2011 .

[7]  Daniel L. Civco,et al.  Artificial Neural Networks for Land-Cover Classification and Mapping , 1993, Int. J. Geogr. Inf. Sci..

[8]  Luc Van Gool,et al.  The Pascal Visual Object Classes (VOC) Challenge , 2010, International Journal of Computer Vision.

[9]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[10]  S. Silvestri,et al.  Mapping salt-marsh vegetation by multispectral and hyperspectral remote sensing , 2006 .

[11]  M. Netto,et al.  An unsupervised method of classifying remotely sensed images using Kohonen self‐organizing maps and agglomerative hierarchical clustering methods , 2008 .

[12]  Patricia Gober,et al.  Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery , 2011, Remote Sensing of Environment.

[13]  Bing Zhang,et al.  A Review of Remote Sensing Image Classification Techniques: the Role of Spatio-contextual Information , 2014 .

[14]  Shengrui Wang,et al.  Image classification algorithm based on the RBF neural network and K-means , 1998 .

[15]  Rebecca C. Smyth,et al.  Mapping coastal environments with lidar and EM on Mustang Island, Texas, U.S. , 2004 .

[16]  Antonio Plaza,et al.  A new deep convolutional neural network for fast hyperspectral image classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[17]  Xing Zhao,et al.  Spectral–Spatial Classification of Hyperspectral Data Based on Deep Belief Network , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[18]  Tong Zhang,et al.  Deep Learning Based Feature Selection for Remote Sensing Scene Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[19]  S. Silvestri,et al.  Hyperspectral remote sensing of salt marsh vegetation, morphology and soil topography , 2003 .

[20]  Anton van den Hengel,et al.  Wider or Deeper: Revisiting the ResNet Model for Visual Recognition , 2016, Pattern Recognit..

[21]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[22]  Roberto Cipolla,et al.  SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Tao Liu,et al.  Object-based classification of wetland vegetation using very high-resolution unmanned air system imagery , 2017 .

[24]  Andrea Taramelli,et al.  Indications of Dynamic Effects on Scaling Relationships Between Channel Sinuosity and Vegetation Patch Size Across a Salt Marsh Platform , 2018, Journal of Geophysical Research: Earth Surface.

[25]  M. Westoby,et al.  ‘Structure-from-Motion’ photogrammetry: A low-cost, effective tool for geoscience applications , 2012 .

[26]  Rasim Latifovic,et al.  Assessment of Convolution Neural Networks for Wetland Mapping with Landsat in the Central Canadian Boreal Forest Region , 2019, Remote. Sens..

[27]  Carlo Gatta,et al.  Unsupervised Deep Feature Extraction for Remote Sensing Image Classification , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Tao Liu,et al.  Deep convolutional neural network training enrichment using multi-view object-based analysis of Unmanned Aerial systems imagery for wetlands classification , 2018 .

[29]  Tao Liu,et al.  Comparing fully convolutional networks, random forest, support vector machine, and patch-based deep convolutional neural networks for object-based wetland mapping using images from small unmanned aircraft system , 2018 .

[30]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[31]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[32]  Stefan W. Maier,et al.  Comparing object-based and pixel-based classifications for mapping savannas , 2011, Int. J. Appl. Earth Obs. Geoinformation.

[33]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[35]  Johannes R. Sveinsson,et al.  Multispectral and Hyperspectral Image Fusion Using a 3-D-Convolutional Neural Network , 2017, IEEE Geoscience and Remote Sensing Letters.

[36]  C. Milesi,et al.  Multi-scale standardized spectral mixture models , 2013 .

[37]  Pi-Fuei Hsieh,et al.  Effect of spatial resolution on classification errors of pure and mixed pixels in remote sensing , 2001, IEEE Trans. Geosci. Remote. Sens..

[38]  Paul M. Mather,et al.  Classification of multisource remote sensing imagery using a genetic algorithm and Markov random fields , 1999, IEEE Trans. Geosci. Remote. Sens..

[39]  Chuyen Nguyen,et al.  Unsupervised Clustering of Multi-Perspective 3D Point Cloud Data in Marshes: A Case Study , 2019, Remote. Sens..

[40]  Tom Spencer,et al.  Assessing seasonal vegetation change in coastal wetlands with airborne remote sensing: an outline methodology , 1998 .

[41]  Xiaomin Li,et al.  Hyperspectral Coastal Wetland Classification Based on a Multiobject Convolutional Neural Network Model and Decision Fusion , 2019, IEEE Geoscience and Remote Sensing Letters.