Demosaicing images in low lighting environments

It is commonly believed that having more white pixels in a color filter array (CFA) will help the demosaicing performance for images collected in low lighting conditions. We present a comparative study to evaluate the performance of demosaicing for images collected in realistic low lighting conditions using two CFAs: the standard Bayer pattern (aka CFA 1.0) and the Kodak CFA 2.0 (RGBW pattern with 50% white pixels). Using a data set containing 10 images collected in low lighting conditions, we observe that having more white pixels does help the demosaicing performance. However, some cautions are needed in quantifying the performance.

[1]  Aleksandra Pizurica,et al.  Processing of Multiresolution Thermal Hyperspectral and Digital Color Data: Outcome of the 2014 IEEE GRSS Data Fusion Contest , 2015, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Chiman Kwan,et al.  Enhancing Mastcam Images for Mars Rover Mission , 2017, ISNN.

[3]  Chiman Kwan,et al.  A Novel Utilization of Image Registration Techniques to Process Mastcam Images in Mars Rover With Applications to Image Fusion, Pixel Clustering, and Anomaly Detection , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[4]  Brian A. Wandell,et al.  A spatial extension of CIELAB for digital color‐image reproduction , 1997 .

[5]  황준.  Image sensor with improved light sensitivity and fabricating method of the same , 2001 .

[6]  J. G. Liu,et al.  Smoothing Filter-based Intensity Modulation : a spectral preserve image fusion technique for improving spatial details , 2001 .

[7]  Oscar C. Au,et al.  Exploitation of inter-color correlation for color image demosaicking , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[8]  Wangmeng Zuo,et al.  COLOR IMAGE DEMOSAICKING VIA DEEP RESIDUAL LEARNING , 2017 .

[9]  Bruno Aiazzi,et al.  Improving Component Substitution Pansharpening Through Multivariate Regression of MS $+$Pan Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[10]  Yücel Altunbasak,et al.  Color plane interpolation using alternating projections , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[11]  Yuzhong Shen,et al.  Assessment of Spatiotemporal Fusion Algorithms for Planet and Worldview Images , 2018, Sensors.

[12]  Lei Zhang,et al.  Color demosaicking by local directional interpolation and nonlocal adaptive thresholding , 2011, J. Electronic Imaging.

[13]  Jianqin Zhou,et al.  On discrete cosine transform , 2011, ArXiv.

[14]  Luciano Alparone,et al.  MTF-tailored Multiscale Fusion of High-resolution MS and Pan Imagery , 2006 .

[15]  William T. Freeman,et al.  Rethinking color cameras , 2014, 2014 IEEE International Conference on Computational Photography (ICCP).

[16]  Jocelyn Chanussot,et al.  A Critical Comparison Among Pansharpening Algorithms , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[17]  Chiman Kwan,et al.  Demosaicing enhancement using pixel-level fusion , 2018, Signal Image Video Process..

[18]  Chiman Kwan,et al.  Demosaicing of Bayer and CFA 2.0 Patterns for Low Lighting Images , 2019, Electronics.

[19]  Masatoshi Okutomi,et al.  Beyond Color Difference: Residual Interpolation for Color Image Demosaicking , 2016, IEEE Transactions on Image Processing.

[20]  Lei Zhang,et al.  Color demosaicking via directional linear minimum mean square-error estimation , 2005, IEEE Transactions on Image Processing.

[21]  Chiman Kwan,et al.  Fusion of themis and TES for accurate Mars surface characterization , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[22]  Chiman Kwan,et al.  Hyperspectral image super-resolution: a hybrid color mapping approach , 2016 .

[23]  Chiman Kwan,et al.  Pansharpening of Mastcam images , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[24]  Chiman Kwan,et al.  Resolution enhancement for hyperspectral images: A super-resolution and fusion approach , 2017, 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[25]  L. Macaire,et al.  Comparison of color demosaicing methods , 2010 .

[26]  J. Bednar,et al.  Alpha-trimmed means and their relationship to median filters , 1984 .

[27]  Chiman Kwan,et al.  Further Improvement of Debayering Performance of RGBW Color Filter Arrays Using Deep Learning and Pansharpening Techniques , 2019, J. Imaging.

[28]  Ning Zhang,et al.  Primary-consistent soft-decision color demosaicking for digital cameras (patent pending) , 2004, IEEE Transactions on Image Processing.

[29]  Alan C. Bovik,et al.  Making a “Completely Blind” Image Quality Analyzer , 2013, IEEE Signal Processing Letters.

[30]  Chiman Kwan,et al.  Debayering RGBW color filter arrays: A pansharpening approach , 2017, 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON).

[31]  Feng Gao,et al.  A Hybrid Color Mapping Approach to Fusing MODIS and Landsat Images for Forward Prediction , 2018, Remote. Sens..

[32]  Yap-Peng Tan,et al.  Color filter array demosaicking: new method and performance measures , 2003, IEEE Trans. Image Process..

[33]  Gwanggil Jeon,et al.  Least-Squares Luma–Chroma Demultiplexing Algorithm for Bayer Demosaicking , 2011, IEEE Transactions on Image Processing.

[34]  Thomas Pock,et al.  Learning joint demosaicing and denoising based on sequential energy minimization , 2016, 2016 IEEE International Conference on Computational Photography (ICCP).

[35]  Chiman Kwan,et al.  A Comparison of Compression Codecs for Maritime and Sonar Images in Bandwidth Constrained Applications , 2019, Comput..

[36]  Frédo Durand,et al.  Deep joint demosaicking and denoising , 2016, ACM Trans. Graph..

[37]  Chiman Kwan,et al.  Blind Quality Assessment of Fused WorldView-3 Images by Using the Combinations of Pansharpening and Hypersharpening Paradigms , 2017, IEEE Geoscience and Remote Sensing Letters.

[38]  Kiyun Yu,et al.  A New Adaptive Component-Substitution-Based Satellite Image Fusion by Using Partial Replacement , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Radu Timofte,et al.  Demosaicing Based on Directional Difference Regression and Efficient Regression Priors , 2016, IEEE Transactions on Image Processing.

[40]  Chiman Kwan,et al.  Perceptually Lossless Compression with Error Concealment for Periscope and Sonar Videos , 2019 .

[41]  J. Astola,et al.  ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS , 2007 .

[42]  Jocelyn Chanussot,et al.  Contrast and Error-Based Fusion Schemes for Multispectral Image Pansharpening , 2014, IEEE Geoscience and Remote Sensing Letters.

[43]  Eric Dubois,et al.  Frequency-domain methods for demosaicking of Bayer-sampled color images , 2005, IEEE Signal Processing Letters.

[44]  Masatoshi Okutomi,et al.  Adaptive Residual Interpolation for Color and Multispectral Image Demosaicking † , 2017, Sensors.