Object Detection Using Adaptive Mask RCNN in Optical Remote Sensing Images

Fast and automatic object detection in remote sensing images is a critical and challenging task for civilian and military applications. Recently, deep learning approaches were introduced to overcome the limitation of traditional object detection methods. In this paper, adaptive mask Region-based Convolutional Network (mask-RCNN) is utilized for multi-class object detection in remote sensing images. Transfer learning, data augmentation, and fine-tuning were adopted to overcome objects scale variability, small size, the density of objects, and the scarcity of annotated remote sensing image. Also, five optimization methods were investigated namely: Adaptive Moment Estimation (Adam), stochastic gradient decent (SGD), adaptive learning rate method (Adelta), Root Mean Square Propagation (RMSprop) and hybrid optimization. In hybrid optimization, the training process begins Adam then switches to SGD when appropriate and vice versa. Also, the behaviour of adaptive mask RCNN was compared to baseline deep object detection methods. Several experiments were conducted on the challenging NWPU-VHR-10 dataset. The hybrid method Adam_SGD acheived the highest Accuracy precision, with 95%. Experimental results showed detection performance in terms of accuracy and intersection over union (IOU) boost of performance up to 6%.

[1]  Baojun Zhao,et al.  Compressed-Domain Ship Detection on Spaceborne Optical Image Using Deep Neural Network and Extreme Learning Machine , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Bin Pan,et al.  Cascade Convolutional Neural Network Based on Transfer-Learning for Aircraft Detection on High-Resolution Remote Sensing Images , 2017, J. Sensors.

[3]  Ashok Samal,et al.  Semi-Automated Road Detection From High Resolution Satellite Images by Directional Morphological Enhancement and Segmentation Techniques , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  Lei Guo,et al.  Weakly Supervised Learning for Target Detection in Remote Sensing Images , 2015, IEEE Geoscience and Remote Sensing Letters.

[5]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Sinan Kalkan,et al.  Localization Recall Precision (LRP): A New Performance Metric for Object Detection , 2018, ECCV.

[7]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Tapas Ranjan Martha,et al.  Segment Optimization and Data-Driven Thresholding for Knowledge-Based Landslide Detection by Object-Based Image Analysis , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[9]  Lin Lei,et al.  Multi-scale object detection in remote sensing imagery with convolutional neural networks , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[10]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[11]  Jungho Im,et al.  ISPRS Journal of Photogrammetry and Remote Sensing , 2022 .

[12]  Naoto Yokoya,et al.  Object Detection Based on Sparse Representation and Hough Voting for Optical Remote Sensing Imagery , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[13]  Junwei Han,et al.  A Survey on Object Detection in Optical Remote Sensing Images , 2016, ArXiv.

[14]  Matthew D. Zeiler.  ADADELTA: An Adaptive Learning Rate Method , 2012, ArXiv.

[15]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[16]  Qing Liu,et al.  Accurate Object Localization in Remote Sensing Images Based on Convolutional Neural Networks , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[17]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[18]  Yanfeng Gu,et al.  Cnn Based Renormalization Method for Ship Detection in Vhr Remote Sensing Images , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[19]  Junwei Han,et al.  Learning Rotation-Invariant Convolutional Neural Networks for Object Detection in VHR Optical Remote Sensing Images , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[20]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[21]  Xu Liu,et al.  Deep Adaptive Proposal Network for Object Detection in Optical Remote Sensing Images , 2018, ArXiv.

[22]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Lei Guo,et al.  Effective and Efficient Midlevel Visual Elements-Oriented Land-Use Classification Using VHR Remote Sensing Images , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.

[26]  Yi Li,et al.  R-FCN: Object Detection via Region-based Fully Convolutional Networks , 2016, NIPS.

[27]  W. Marsden.  I and J , 2012 .

[28]  Menglong Yan,et al.  Automatic Ship Detection in Remote Sensing Images from Google Earth of Complex Scenes Based on Multiscale Rotation Dense Feature Pyramid Networks , 2018, Remote. Sens..

[29]  Jaewook Jung,et al.  Building Extraction from Satellite Images Using Mask R-CNN with Building Boundary Regularization , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Lin Lei,et al.  An enhanced deep convolutional neural network for densely packed objects detection in remote sensing images , 2017, 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP).

[31]  Ali Ozgun Ok,et al.  Automated detection of buildings from single VHR multispectral images using shadow information and graph cuts , 2013 .

[32]  Zhiguo Jiang,et al.  Inshore Ship Detection Based on Mask R-CNN , 2018, IGARSS 2018 - 2018 IEEE International Geoscience and Remote Sensing Symposium.

[33]  Richard Socher,et al.  Improving Generalization Performance by Switching from Adam to SGD , 2017, ArXiv.

[34]  Bo Li,et al.  Ship Detection in High-Resolution Optical Imagery Based on Anomaly Detector and Local Shape Feature , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[36]  Qi Tian,et al.  Feature representation for statistical-learning-based object detection: A review , 2015, Pattern Recognit..

[37]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[38]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Awais Ahmad,et al.  Real-Time Big Data Analytical Architecture for Remote Sensing Application , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[40]  Jie Wang,et al.  Transferring Pre-Trained Deep CNNs for Remote Scene Classification with General Features Learned from Linear PCA Network , 2017, Remote. Sens..

[41]  Naif Alajlan,et al.  Deep Learning Approach for Car Detection in UAV Imagery , 2017, Remote. Sens..

[42]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Ting Liu,et al.  Recent advances in convolutional neural networks , 2015, Pattern Recognit..