Spectral Constrained Residual Attention Network for Hyperspectral Pansharpening

Deep learning methods have been widely used in the task of hyperspectral pansharpening. However, most of these methods regard the Panchromatic (PAN) image as a kind of auxiliary information, which is mainly used as spatial details to add on the hyperspectral image (HSI) after processing. Obviously, this kind of methods utilize the PAN image insufficiently, resulting in the imbalance of spatial preservation and spatial preservation. In this paper, a spectral constrained residual attention network (SCRAN) is proposed by using the PAN image as the foundation of the pansharpening task and concerning on the spectral and spatial learning. The proposed SCRAN method consists of three parts: a spectral feature extraction net, an attention spatial residual net and a spectral reconstruction net. A spectral constrained loss function is designed to enhance the spectral learning ability of SCRAN. Additionally, in SCRAN, a deep back-projection network (DBPN) is operated to upsample the HSI, and the histogram matching is applied to the PAN image to make it closer to the HSI in terms of spectral bands.