Automated terrain feature identification from remote sensing imagery: a deep learning approach

ABSTRACT Terrain feature detection is a fundamental task in terrain analysis and landscape scene interpretation. Discovering where a specific feature (i.e. sand dune, crater, etc.) is located and how it evolves over time is essential for understanding landform processes and their impacts on the environment, ecosystem, and human population. Traditional induction-based approaches are challenged by their inefficiency for generalizing diverse and complex terrain features as well as their performance for scalable processing of the massive geospatial data available. This paper presents a new deep learning (DL) approach to support automatic detection of terrain features from remotely sensed images. The novelty of this work lies in: (1) a terrain feature database containing 12,000 remotely sensed images (1,000 original images and 11,000 derived images from data augmentation) that supports data-driven model training and new discovery; (2) a DL-based object detection network empowered by ensemble learning and deep and deeper convolutional neural networks to achieve high-accuracy object detection; and (3) fine-tuning the model’s characteristics and behaviors to identify the best combination of hyperparameters and other network factors. The introduction of DL into geospatial applications is expected to contribute significantly to intelligent terrain analysis, landscape scene interpretation, and the maturation of spatial data science.

[1]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[2]  Sanja Fidler,et al.  Monocular 3D Object Detection for Autonomous Driving , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Brian Tomaszewski,et al.  Refugee Situation Awareness: Camps and Beyond☆ , 2015 .

[4]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Yong Dou,et al.  Airport Detection on Optical Satellite Images Using Deep Convolutional Neural Networks , 2017, IEEE Geoscience and Remote Sensing Letters.

[7]  Sergio Guadarrama,et al.  Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[10]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[12]  Andy Liaw,et al.  Classification and Regression by randomForest , 2007 .

[13]  Ute Christina Herzfeld,et al.  Atlas of Antarctica: Topographic Maps from Geostatistical Analysis of Satellite Radar Altimeter Data , 2012 .

[14]  Brian Becker,et al.  Atlas of Antarctica: Topographic Maps From Geostatistical Analysis of Satellite Radar Altimeter Data , 2006, Cartogr. Int. J. Geogr. Inf. Geovisualization.

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Ping Tang,et al.  Feature significance-based multibag-of-visual-words model for remote sensing image scene classification , 2016 .

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Wen Yang,et al.  High-resolution satellite scene classification using a sparse coding based multiple feature combination , 2012 .

[19]  Dong Xu,et al.  Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey , 2018, IEEE Signal Processing Magazine.

[20]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Christian Wolf,et al.  Sequential Deep Learning for Human Action Recognition , 2011, HBU.

[22]  Proceedings of the 1st Workshop on Artificial Intelligence and Deep Learning for Geographic Knowledge Discovery , 2017, GeoAI@SIGSPATIAL.

[23]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[25]  Zhao Lin,et al.  A modified faster R-CNN based on CFAR algorithm for SAR ship detection , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[26]  Huanxin Zou,et al.  Toward Fast and Accurate Vehicle Detection in Aerial Images Using Coupled Region-Based Convolutional Neural Networks , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[27]  Constantin Vertan,et al.  The use of deep learning in image segmentation, classification and detection , 2016, ArXiv.

[28]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[29]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[30]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[32]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[33]  Larry S. Davis,et al.  An Analysis of Scale Invariance in Object Detection - SNIP , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[34]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[35]  Zhiguo Jiang,et al.  Chimney and condensing tower detection based on faster R-CNN in high resolution remote sensing images , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[36]  Lei Guo,et al.  Exploring Hierarchical Convolutional Features for Hyperspectral Image Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[37]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[38]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[39]  Jungho Im,et al.  Forest biomass estimation from airborne LiDAR data using machine learning approaches , 2012 .

[40]  Junwei Han,et al.  Duplex Metric Learning for Image Set Classification , 2018, IEEE Transactions on Image Processing.

[41]  Feiping Nie,et al.  Robust Object Co-Segmentation Using Background Prior , 2018, IEEE Transactions on Image Processing.

[42]  Quoc V. Le,et al.  AutoAugment: Learning Augmentation Policies from Data , 2018, ArXiv.

[43]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[44]  Myung-Cheol Roh,et al.  Refining faster-RCNN for accurate object detection , 2017, 2017 Fifteenth IAPR International Conference on Machine Vision Applications (MVA).

[45]  Shiming Xiang,et al.  Automatic Road Detection and Centerline Extraction via Cascaded End-to-End Convolutional Neural Network , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[46]  Bin Zhou,et al.  Recognizing terrain features on terrestrial surface using a deep learning model: an example with crater detection , 2017, GeoAI@SIGSPATIAL.

[47]  Christopher Leckie,et al.  High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning , 2016, Pattern Recognit..

[48]  Biao Yang,et al.  On Road Vehicle Detection Using an Improved Faster RCNN Framework with Small-Size Region Up-Scaling Strategy , 2017, PSIVT Workshops.

[49]  Farid Melgani,et al.  A Convolutional Neural Network Approach for Assisting Avalanche Search and Rescue Operations with UAV Imagery , 2017, Remote. Sens..