Convolutional Neural Networks with Deep Supervised Feature Learning for Remote Sensing Scene Classification

State-of-the-art remote sensing scene classification methods employ different Convolutional 1 Neural Network architectures for achieving very high classification performance. A trait shared 2 by the majority of these methods is that the class associated with each example is ascertained by 3 examining the activations of the last fully connected layer, and the networks are trained to minimize 4 the cross-entropy between predictions extracted from this layer and ground-truth annotations. In 5 this work, we extend this paradigm by introducing an additional output branch which maps the 6 inputs to low dimensional representations, effectively extracting additional feature representations 7 of the inputs. The proposed model imposes additional distance constrains on these representations 8 with respect to identified class representatives, in addition to the traditional categorical cross-entropy 9 between predictions and ground-truth. By extending the typical cross-entropy loss function with 10 a distance learning function, our proposed approach achieves significant gains across a wide set of 11 benchmark datasets in terms of classification, while providing additional evidence related to class 12 membership and classification confidence. 13

[1]  Zhen Ye,et al.  Deep Metric Learning Based on Scalable Neighborhood Components for Remote Sensing Scene Characterization , 2020, IEEE Transactions on Geoscience and Remote Sensing.

[2]  Panagiotis Tsakalides,et al.  Survey of Deep-Learning Approaches for Remote Sensing Observation Enhancement , 2019, Sensors.

[3]  Jian Wang,et al.  Deep Metric Learning with Angular Loss , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Seong-Whan Lee,et al.  Coarse-to-Fine Deep Metric Learning for Remote Sensing Image Retrieval , 2020, Remote. Sens..

[5]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[6]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[9]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Hasan Şakir Bilge,et al.  Deep Metric Learning: A Survey , 2019, Symmetry.

[11]  Ping Zhong,et al.  Diversity-Promoting Deep Structural Metric Learning for Remote Sensing Scene Classification , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Mohamed Farah,et al.  Remote Sensing Scene Classification Using Convolutional Features and Deep Forest Classifier , 2019, IEEE Geoscience and Remote Sensing Letters.

[14]  Lei Guo,et al.  When Deep Learning Meets Metric Learning: Remote Sensing Image Scene Classification via Learning Discriminative CNNs , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Xiaoqiang Lu,et al.  Remote Sensing Image Scene Classification: Benchmark and State of the Art , 2017, Proceedings of the IEEE.

[16]  Kihyuk Sohn,et al.  Improved Deep Metric Learning with Multi-class N-pair Loss Objective , 2016, NIPS.

[17]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[18]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[19]  Rasmus Fensholt,et al.  Remote Sensing , 2008, Encyclopedia of GIS.

[20]  Francisco Herrera,et al.  A Tutorial on Distance Metric Learning: Mathematical Foundations, Algorithms and Software , 2018, ArXiv.

[21]  Jun Zhang,et al.  A Multi-Scale Approach for Remote Sensing Scene Classification Based on Feature Maps Selection and Region Representation , 2019, Remote. Sens..

[22]  Shawn D. Newsam,et al.  Bag-of-visual-words and spatial extensions for land-use classification , 2010, GIS '10.

[23]  Haokui Zhang,et al.  Deep learning for remote sensing image classification: A survey , 2018, WIREs Data Mining Knowl. Discov..

[24]  D. Lu,et al.  Remote Sensing Image Classification , 2011 .

[25]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Gui-Song Xia,et al.  AID: A Benchmark Data Set for Performance Evaluation of Aerial Scene Classification , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[27]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.