A temporal group attention approach for multitemporal multisensor crop classification

Abstract Multitemporal observation capability is of great significance in crop classification. Benefiting from time-intensive multitemporal multisensor RS data, we can obtain the fine temporal profiles of crops, and how to dig out the subtle differences of phenological profiles between crops is of great significance. In this paper, we propose a Temporal-Group-Attention classifier to distinguish subtle differences of crops phenology, and the TGA(Temporal-Group-Attention) modules imitate the information selection mechanism of cognitive neuroscience to highlight differences between categories and ignore their similarities. After first obtaining the group sequence features of multitemporal RS images, then we use TGA modules to select time periods with large differences in phenology. And the TGA mechanism utilizes a “feature recalibration” strategy to highlight phenological differences, which firstly performs “group squeeze”, then “group excitation”, and finally performs “group reweighting”. In experiments, the proposed network is illustrated on a multitemporal dataset with sequence length of 65, in which the samples from Sentinel-2A/B and Landsat-8 are acquired over Central Valley of California throughout 2018. Then, we perform multiresolution fusion for the sensors with different spatial resolutions. Finally, the experimental results show that the proposed approach can achieve better classification accuracies over the categories with similar phenological profiles, compared with traditional methods and regular group convolution network without attention mechanism.