Deep Panchromatic Image Guided Residual Interpolation For Multispectral Image Demosaicking

Snapshot multispectral imaging based on multispectral filter arrays (MSFA) has gained popularity recently for its size and speed. To process these multispectral images, demosaicking is the most crucial and challenging step to reduce artifacts in both spatial and spectral domain. In this work, a novel ResNet based deep learning model is first proposed to reconstruct the full-resolution panchromatic image from MSFA mosaic image. Then, the reconstructed deep panchromatic image (DPI) is deployed as the guide to recover the full-resolution multispectral image using a two-pass guided residual interpolation method. Experiment results demonstrate that the proposed method outperforms the state-of-the-art conventional and deep learning demosaicking methods both qualitatively and quantitatively.