Deep multi-feature fusion network for remote sensing images

ABSTRACT Learning discriminative and robust features is crucial in remote sensing image processing. Many of the currently used approaches are based on Convolutional Neural Networks (CNNs). However, such approaches may not effectively capture various different semantic objects of remote sensing images. To overcome this limitation, we propose a novel end-to-end deep multi-feature fusion network (DMFN). DMFN combines two different deep architecture branches for feature representations; the global and local branch. The global branch, which consists of three losses, is used to learn discriminative features from the whole image. The local branch is then used in the partitioning of the entire image into multiple strips in order to obtain local features. The two branches are then combined, used to learn fusion feature representations for the image. The proposed method is an end-to-end framework during training. Comprehensive validation experiments on two public datasets indicate that relative to existing deep learning approaches, this strategy is superior for both retrieval and classification tasks.

[1]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[5]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[7]  Zhenfeng Shao,et al.  Region Convolutional Features for Multi-Label Remote Sensing Image Retrieval , 2018, ArXiv.

[8]  Florent Perronnin,et al.  Fisher Kernels on Visual Vocabularies for Image Categorization , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[9]  Lei Guo,et al.  Remote Sensing Image Scene Classification Using Bag of Convolutional Features , 2017, IEEE Geoscience and Remote Sensing Letters.

[10]  Anil M. Cheriyadat,et al.  Unsupervised Feature Learning for Aerial Scene Classification , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[12]  Gang Ma,et al.  Multi-feature fusion deep networks , 2016, Neurocomputing.

[13]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[14]  Min Wang,et al.  Remote Sensing Image Retrieval by Scene Semantic Matching , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Wei Xiong,et al.  A Discriminative Feature Learning Approach for Remote Sensing Image Retrieval , 2019, Remote. Sens..

[17]  Fang Liu,et al.  Unsupervised Deep Feature Learning for Remote Sensing Image Retrieval , 2018, Remote. Sens..

[18]  Peter H. N. de With,et al.  Aggregated Deep Local Features for Remote Sensing Image Retrieval , 2019, Remote. Sens..

[19]  Peng Li,et al.  Region-Wise Deep Feature Representation for Remote Sensing Images , 2018, Remote. Sens..

[20]  Xiaodong Mu,et al.  Remote sensing image scene classification based on generative adversarial networks , 2018 .

[21]  Jianbo Liu,et al.  An improved Bag-of-Words framework for remote sensing image retrieval in large-scale image databases , 2015, Int. J. Digit. Earth.