COLOR: Cycling, Offline Learning, and Online Representation Framework for Airport and Airplane Detection Using GF-2 Satellite Images

Monitoring airports using remote sensing imagery require us to first detect the airports and then perform airplane detection. Detecting airports and airplanes with large-scale remote sensing imagery are significant and challenging tasks in the field of remote sensing. Although many detection algorithms have been developed for detecting airports and airplanes in remote sensing imagery, the efficiency of the processing does not meet the needs of real applications in large-scale remote sensing imagery. In recent years, deep learning techniques, such as deep convolutional neural networks (DCNNs), have achieved great progress in image recognition. However, training a DCNN needs a large number of training examples to accurately fit the data distribution. Annotating training examples in large-scale remote sensing imagery is time-consuming, which makes the pipeline inefficient. In this article, to overcome the above two weaknesses, we propose a novel cycling data-driven framework for efficient and robust airport localization and airplane detection. The proposed method consists of three modules: cycling by example refinement (C), offline learning (OL), and online representation (OR), namely cycling, offline learning, and online representation (COLOR). The OR module is a coarse-to-fine cascaded convolutional neural network, which is used to detect airports and airplanes. The example refinement (ER) module implements the cycling and makes use of the unlabeled remote sensing images and the corresponding predictions obtained by the OR module, to generate training examples. The OL module aims to use the training examples from the ER module to update the OR module, to further improve the performance. The whole workflow involves COLOR. The COLOR framework was used to detect airplanes and airports in 512 large-scale Gaofen-2 (GF-2) remote sensing images with 29 $200\times27$ 620 pixels. The results showed that the proposed method obtained a mean average precision (mAP) of 88.32% for the airplane detection. In addition due to the proposed coarse-to-fine cascaded OR module the proposed method is much faster than the traditional approaches in real-world applications.

[1]  Scott Lundberg,et al.  Checkpoint Ensembles: Ensemble Methods from a Single Training Process , 2017, ArXiv.

[2]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[3]  Yakoub Bazi,et al.  Asymmetric Adaptation of Deep Features for Cross-Domain Classification in Remote Sensing Imagery , 2018, IEEE Geoscience and Remote Sensing Letters.

[4]  Xiaogang Wang,et al.  Deep Convolutional Network Cascade for Facial Point Detection , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Rasoul Karimi,et al.  Active Learning for Recommender Systems , 2015, KI - Künstliche Intelligenz.

[6]  Zexuan Zhu,et al.  Computational intelligence in optical remote sensing image processing , 2018, Appl. Soft Comput..

[7]  Jiebo Luo,et al.  DOTA: A Large-Scale Dataset for Object Detection in Aerial Images , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Chao Li,et al.  Active multi-kernel domain adaptation for hyperspectral image classification , 2017, Pattern Recognit..

[9]  Bo Du,et al.  Weakly Supervised Learning Based on Coupled Convolutional Neural Networks for Aircraft Detection , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[11]  Jian Sun,et al.  Face Alignment at 3000 FPS via Regressing Local Binary Features , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Yuning Jiang,et al.  Extensive Facial Landmark Localization with Coarse-to-Fine Convolutional Network Cascade , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[13]  Shukui Bo,et al.  Region-based airplane detection in remotely sensed imagery , 2010, 2010 3rd International Congress on Image and Signal Processing.

[14]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Stefan Schaal,et al.  Incremental Online Learning in High Dimensions , 2005, Neural Computation.

[16]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Gregory D. Hager,et al.  Semantic Stereo for Incidental Satellite Images , 2018, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[18]  Xiaobin Li,et al.  Airplane detection using convolutional neural networks in a coarse-to-fine manner , 2017, 2017 IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[19]  Lifei Wei,et al.  Mini-UAV-Borne Hyperspectral Remote Sensing: From Observation and Processing to Applications , 2018, IEEE Geoscience and Remote Sensing Magazine.

[20]  Xinyu Wang,et al.  Satellite-ground integrated destriping network: A new perspective for EO-1 Hyperion and Chinese hyperspectral satellite datasets , 2020 .

[21]  Wei Li,et al.  Robust airplane detection in satellite images , 2011, 2011 18th IEEE International Conference on Image Processing.

[22]  Francis R. Bach,et al.  Online Learning for Latent Dirichlet Allocation , 2010, NIPS.

[23]  Hao Dou,et al.  Airplane detection based on fusion framework by combining saliency model with Deep Convolutional Neural Networks , 2018, International Symposium on Multispectral Image Processing and Pattern Recognition.

[24]  Lorenzo Bruzzone,et al.  Domain Adaptation for the Classification of Remote Sensing Data: An Overview of Recent Advances , 2016, IEEE Geoscience and Remote Sensing Magazine.

[25]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[26]  Hui Wu,et al.  Fast aircraft detection in satellite images based on convolutional neural networks , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[27]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[29]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[30]  Zhang Hao,et al.  Incremental and Online Learning Algorithm for Regression Least Squares Support Vector Machine , 2006 .

[31]  Lei Zhang,et al.  Fine-Tuning Convolutional Neural Networks for Biomedical Image Analysis: Actively and Incrementally , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Hongping Cai,et al.  Airplane detection in remote sensing image with a circle-frequency filter , 2005, International Conference on Space Information Technology.

[33]  Ali Farhadi,et al.  YOLOv3: An Incremental Improvement , 2018, ArXiv.

[34]  Larry S. Davis,et al.  Soft-NMS — Improving Object Detection with One Line of Code , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[35]  Shawki Areibi,et al.  Domain Adaptation Using Representation Learning for the Classification of Remote Sensing Images , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[36]  Dawei Zai,et al.  Rotation-and-scale-invariant airplane detection in high-resolution satellite images based on deep-Hough-forests , 2016 .

[37]  Zhenwei Shi,et al.  Airplane detection based on rotation invariant and sparse coding in remote sensing images , 2014 .

[38]  Amir Babaeian,et al.  Airplane detection and tracking using wavelet features and SVM classifier , 2009, 2009 41st Southeastern Symposium on System Theory.

[39]  Jing Huang,et al.  DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[40]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.