Hyperspectral Image Super-Resolution With a Mosaic RGB Image

Recently, many hyperspectral (HS) image super-resolution methods that merge a low spatial resolution HS image and a high spatial resolution three-channel RGB image have been proposed in spectral imaging. A largely ignored fact is that most existing commercial RGB cameras capture high resolution images by a single CCD/CMOS sensor equipped with a color filter array. In this paper, we account for the common imaging mechanism of commercial RGB cameras, and propose to use a mosaic RGB image for HS image super-resolution, which prevents demosaicing error and thus its propagation into the HS image super-resolution results. We design a proper non-local low-rank regularization to exploit the intrinsic properties—rich self-repeating patterns and high correlation across spectra—within HS images of natural scenes, and formulate the HS image super-resolution task into a variational optimization problem, which can be efficiently solved via the alternating direction method of multipliers. The effectiveness of the proposed method has been evaluated on two benchmark data sets, demonstrating that the proposed method can provide substantial improvement over the current state-of-the-art HS image super-resolution methods without considering the mosaicing effect. Finally, we show that our method can also perform well in the real capture system.