Deep demosaicking for multispectral filter arrays.

We propose a novel demosaicking method for multispectral filter arrays based on a deep convolutional neural network. The proposed method first interpolates mosaicked multispectral images utilizing a bilinear approach, then applies a residual network to initial demosaicked images. The residual network consists of various three-dimensional convolutional layers and a rectified linear unit for describing the features of a multispectral data cube. Experimental results reveal that the proposed method outperforms conventional demosaicking methods.

[1]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[2]  Nikos Komodakis,et al.  Wide Residual Networks , 2016, BMVC.

[3]  T. Aach,et al.  A Color Filter Array Based Multispectral Camera , 2006 .

[4]  Shigeo Kato,et al.  Demosaicking for multispectral images based on vectorial total variation , 2016 .

[5]  J. Chanussot,et al.  Hyperspectral Remote Sensing Data Analysis and Future Challenges , 2013, IEEE Geoscience and Remote Sensing Magazine.

[6]  Shree K. Nayar,et al.  Generalized Assorted Pixel Camera: Postcapture Control of Resolution, Dynamic Range, and Spectrum , 2010, IEEE Transactions on Image Processing.

[7]  Masahiro Ishikawa,et al.  Joint optimization of multispectral filter arrays and demosaicking for pathological images , 2018, 1807.01385.

[8]  Masahiro Ishikawa,et al.  Multispectral filter array and demosaicking for pathological images , 2015, 2015 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA).

[9]  Shigeo Kato,et al.  Multispectral demosaicking algorithm based on inter-channel correlation , 2014, 2014 IEEE Visual Communications and Image Processing Conference.

[10]  Ludovic Macaire,et al.  Multispectral Demosaicing Using Pseudo-Panchromatic Image , 2017, IEEE Transactions on Computational Imaging.

[11]  Yoshio Hayasaki,et al.  Multispectral filter array design without training images , 2017 .

[12]  Kai-Lung Hua,et al.  DeepDemosaicking: Adaptive Image Demosaicking via Multiple Deep Fully Convolutional Networks , 2018, IEEE Transactions on Image Processing.

[13]  Ying Li,et al.  Spectral-Spatial Classification of Hyperspectral Imagery with 3D Convolutional Neural Network , 2017, Remote. Sens..

[14]  Angshul Majumdar,et al.  Compressive sensing multi-spectral demosaicing from single sensor architecture , 2014, 2014 IEEE China Summit & International Conference on Signal and Information Processing (ChinaSIP).

[15]  Henrique S. Malvar,et al.  High-quality linear interpolation for demosaicing of Bayer-patterned color images , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[16]  Jie Jia,et al.  Fourier Spectral Filter Array for Optimal Multispectral Imaging , 2016, IEEE Transactions on Image Processing.

[17]  Masahiro Ishikawa,et al.  Demosaicking Method for Multispectral Images Based on Spatial Gradient and Inter-channel Correlation , 2016, ICISP.

[18]  Masatoshi Okutomi,et al.  A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor , 2015, IEEE Transactions on Image Processing.

[19]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[20]  Guolan Lu,et al.  Medical hyperspectral imaging: a review , 2014, Journal of biomedical optics.

[21]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.