Hyperspectral image super-resolution with spectral–spatial network

ABSTRACT The super-resolution problem for hyperspectral images is currently one of the most challenging topics in remote sensing. Increasingly effective methods have been presented to solve this ill-posed problem under certain circumstances. In this article, we propose a new approach named the spectral–spatial network (SSN), which can effectively increase spatial resolution while keeping spectral information. The SSN consists of two sections: a spatial section and a spectral section that contribute to enhancing spatial resolution and preserving spectral information, respectively. The spatial section is proposed to learn end-to-end mapping between single-band images, from low-resolution and high-resolution hyperspectral images. In this section, we enhance the traditional sub-pixel convolutional layer by adding a maximum variance principle that can realize nonlinear fitting through piecewise linearization. The spectral section aims to fine-tune spectral caves to keep the spectral signature with a spectral angle error loss function. In order to make the SSN converge quickly, we also develop a corresponding three-step training method. The experimental results on two databases, with both indoor and outdoor scenes, show that our proposed method performs better than the existing state-of-the-art methods.