Reconstruction of Spectral Reflectance Based on Fusion Convolution Neural Network

An improved method for spectral reflectance reconstruction from the RGB response of the digital camera is proposed by deep convolution neural network. The proposed method learns a fusion mapping theory that represents the basic characteristics of spectral reflectance, including spectral nonlinear mapping and spatial similarity. The deep residual network greatly deepens the network structure and improves the performance of the model, enabling the recovery network to make full use of the sample structure information to accurately recover the spectral reflectance. The performance of the method is compared with several existing methods in the cases of the same experimental data set. Experimental results show that the proposed method has the best accuracy in estimating spectral reflectance and colorimetric values in most cases in comparison with existing methods.

[1]  Antonio Robles-Kelly,et al.  Single Image Spectral Reconstruction for Multimedia Applications , 2015, ACM Multimedia.

[2]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[3]  Yoichi Sato,et al.  From RGB to Spectrum for Natural Scenes via Manifold-Based Mapping , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[4]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[5]  Boaz Arad,et al.  Sparse Recovery of Hyperspectral Signal from Natural RGB Images , 2016, ECCV.

[6]  Tao Zhang,et al.  Joint Camera Spectral Sensitivity Selection and Hyperspectral Image Recovery , 2018, ECCV.

[7]  Michael S. Brown,et al.  Training-Based Spectral Reconstruction from a Single RGB Image , 2014, ECCV.

[8]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Junfeng Yang,et al.  PanNet: A Deep Network Architecture for Pan-Sharpening , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Lei Zhang,et al.  Centralized sparse representation for image restoration , 2011, 2011 International Conference on Computer Vision.

[11]  Joost van de Weijer,et al.  Adversarial Networks for Spatial Context-Aware Spectral Image Reconstruction from RGB , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).