Color Restoration of RGBN Multispectral Filter Array Sensor Images Based on Spectral Decomposition

A multispectral filter array (MSFA) image sensor with red, green, blue and near-infrared (NIR) filters is useful for various imaging applications with the advantages that it obtains color information and NIR information simultaneously. Because the MSFA image sensor needs to acquire invisible band information, it is necessary to remove the IR cut-offfilter (IRCF). However, without the IRCF, the color of the image is desaturated by the interference of the additional NIR component of each RGB color channel. To overcome color degradation, a signal processing approach is required to restore natural color by removing the unwanted NIR contribution to the RGB color channels while the additional NIR information remains in the N channel. Thus, in this paper, we propose a color restoration method for an imaging system based on the MSFA image sensor with RGBN filters. To remove the unnecessary NIR component in each RGB color channel, spectral estimation and spectral decomposition are performed based on the spectral characteristics of the MSFA sensor. The proposed color restoration method estimates the spectral intensity in NIR band and recovers hue and color saturation by decomposing the visible band component and the NIR band component in each RGB color channel. The experimental results show that the proposed method effectively restores natural color and minimizes angular errors.

[1]  Sabine Süsstrunk,et al.  Illuminant estimation and detection using near-infrared , 2009, Electronic Imaging.

[2]  Sabine Süsstrunk,et al.  Color image dehazing using the near-infrared , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[3]  Zhenyue Chen,et al.  RGB-NIR multispectral camera. , 2014, Optics express.

[4]  Ajay Kumar,et al.  Personal Authentication Using Hand Vein Triangulation and Knuckle Shape , 2009, IEEE Transactions on Image Processing.

[5]  Fanzhi Kong,et al.  Color image watermarking algorithm based on HSI color space , 2010, 2010 2nd International Conference on Industrial and Information Systems.

[6]  Kurt C. Lawrence,et al.  Multispectral imaging system with interchangeable filter design , 2010 .

[7]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  David Lee,et al.  Dual Aperture Photography: Image and Depth from a Mobile Camera , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[9]  Christine Pohl,et al.  Multisensor image fusion in remote sensing: concepts, methods and applications , 1998 .

[10]  J. H. Park,et al.  Spatially adaptive multi-resolution multispectral image fusion , 2004 .

[11]  Sabine Süsstrunk,et al.  Colouring the Near-Infrared , 2008, CIC.

[12]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[13]  Dirk Hertel,et al.  A low-cost VIS-NIR true color night vision video system based on a wide dynamic range CMOS imager , 2009, 2009 IEEE Intelligent Vehicles Symposium.

[14]  Erik Reinhard,et al.  Color Transfer between Images , 2001, IEEE Computer Graphics and Applications.

[15]  Masatoshi Okutomi,et al.  N-to-SRGB Mapping for Single-Sensor Multispectral Imaging , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[16]  Moon Gi Kang,et al.  Color Restoration for Infrared Cutoff Filter Removed RGBN Multispectral Filter Array Image Sensor , 2015, VISAPP.

[17]  D H Brainard,et al.  Bayesian color constancy. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[18]  Sabine Süsstrunk,et al.  A novel compressive sensing approach to simultaneously acquire color and near-infrared images on a single sensor , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[19]  Masatoshi Okutomi,et al.  Simultaneous capturing of RGB and additional band images using hybrid color filter array , 2014, Electronic Imaging.

[20]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Dong Yi,et al.  Face Matching Between Near Infrared and Visible Light Images , 2007, ICB.

[22]  Henry R. Kang.  Computational Color Technology , 2006 .

[23]  Graham D. Finlayson,et al.  Improving gamut mapping color constancy , 2000, IEEE Trans. Image Process..

[24]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..

[25]  B. Funt,et al.  Diagonal versus affine transformations for color correction. , 2000, Journal of the Optical Society of America. A, Optics, image science, and vision.

[26]  Na Yang,et al.  A near-infrared imaging method for capturing the interior of a vehicle through windshield , 2010, 2010 IEEE Southwest Symposium on Image Analysis & Interpretation (SSIAI).

[27]  Y. Inaba,et al.  A Day and Night Vision MOS Imager With Robust Photonic-Crystal-Based RGB-and-IR , 2008, IEEE Transactions on Electron Devices.

[28]  Takahiro Okabe,et al.  Image Enhancement of Low-light Scenes with Near-infrared Flash Images , 2009, IPSJ Trans. Comput. Vis. Appl..

[29]  Shengcai Liao,et al.  Illumination Invariant Face Recognition Using Near-Infrared Images , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Sabine Süsstrunk,et al.  Material Classification Using Color and NIR Images , 2009, CIC.

[31]  Sabine Süsstrunk,et al.  Designing color filter arrays for the joint capture of visible and near-infrared images , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[32]  H. S. Wolff,et al.  iRun: Horizontal and Vertical Shape of a Region-Based Graph Compression , 2022, Sensors.