In this paper, we propose a remote-sensing scene-classification method based on vision transformers. These types of networks, which are now recognized as state-of-the-art models in natural language processing, do not rely on convolution layers as in standard convolutional neural networks (CNNs). Instead, they use multihead attention mechanisms as the main building block to derive long-range contextual relation between pixels in images. In a first step, the images under analysis are divided into patches, then converted to sequence by flattening and embedding. To keep information about the position, embedding position is added to these patches. Then, the resulting sequence is fed to several multihead attention layers for generating the final representation. At the classification stage, the first token sequence is fed to a softmax classification layer. To boost the classification performance, we explore several data augmentation strategies to generate additional data for training. Moreover, we show experimentally that we can compress the network by pruning half of the layers while keeping competing classification accuracies. Experimental results conducted on different remote-sensing image datasets demonstrate the promising capability of the model compared to state-of-the-art methods. Specifically, Vision Transformer obtains an average classification accuracy of 98.49%, 95.86%, 95.56% and 93.83% on Merced, AID, Optimal31 and NWPU datasets, respectively. While the compressed version obtained by removing half of the multihead attention layers yields 97.90%, 94.27%, 95.30% and 93.05%, respectively.
[1]
Geoffrey E. Hinton,et al.
Distilling the Knowledge in a Neural Network
,
2015,
ArXiv.
[2]
Lukasz Kaiser,et al.
Attention is All you Need
,
2017,
NIPS.
[3]
A. Skidmore,et al.
Use of remote sensing and GIS for sustainable land management
,
1997
.
[4]
Graham W. Taylor,et al.
Improved Regularization of Convolutional Neural Networks with Cutout
,
2017,
ArXiv.
[5]
Georg Heigold,et al.
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
,
2021,
ICLR.
[6]
Hongyi Zhang,et al.
mixup: Beyond Empirical Risk Minimization
,
2017,
ICLR.
[7]
Kurt Keutzer,et al.
Visual Transformers: Token-based Image Representation and Processing for Computer Vision
,
2020,
ArXiv.
[8]
Song Han,et al.
Deep Compression: Compressing Deep Neural Network with Pruning, Trained Quantization and Huffman Coding
,
2015,
ICLR.
[9]
Ashish Vaswani,et al.
Stand-Alone Self-Attention in Vision Models
,
2019,
NeurIPS.
[10]
Liang Chen,et al.
GAN Augmentation: Augmenting Training Data using Generative Adversarial Networks
,
2018,
ArXiv.