Tunable color correction for noisy images

Abstract. Color correction is one of the most essential camera imaging operations that transforms a camera-specific RGB color space to a standard color space, typically the XYZ or the sRGB color space. Linear color correction (LCC) and polynomial color correction (PCC) are two widely used methods; they perform the color space transformation using a color correction matrix. Owing to the use of high-order terms, PCC generally achieves lower colorimetric errors than LCC. However, PCC amplifies noise more severely than LCC. Consequently, for noisy images, there exists a trade-off between LCC and PCC regarding color fidelity and noise amplification. We propose a color correction framework called tunable color correction (TCC) that enables us to tune the color correction matrix between the LCC and the PCC models. We also derive a mean squared error calculation model of PCC that enables us to select the best trade-off balance in the TCC framework. We experimentally demonstrate that TCC effectively balances the trade-off for noisy images and outperforms LCC and PCC. We also generalize TCC to multispectral cases and demonstrate its effectiveness by taking the color correction for an RGB-near-infrared sensor as an example.

[1]  Raja Bala,et al.  Design and Optimization of Color Lookup Tables on a Simplex Topology , 2012, IEEE Transactions on Image Processing.

[2]  Huaping Liu,et al.  Hue constrained matrix optimization for preferred color reproduction , 2012, J. Electronic Imaging.

[3]  Walter Gish,et al.  Camera Color Correction Using Two-Dimensional Transforms , 2013, Color Imaging Conference.

[4]  Mark S. Drew,et al.  Constrained least-squares regression in color spaces , 1997, J. Electronic Imaging.

[5]  Masatoshi Okutomi,et al.  Single-Sensor RGB-NIR Imaging: High-Quality System Design and Prototype Implementation , 2019, IEEE Sensors Journal.

[6]  Masatoshi Okutomi,et al.  N-to-SRGB Mapping for Single-Sensor Multispectral Imaging , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[7]  Maryam M. Darrodi,et al.  The Alternating Least Squares Technique for Nonuniform Intensity Color Correction , 2015 .

[8]  James Lee Hafner,et al.  Performing color space conversions with three-dimensional linear interpolation , 1995, J. Electronic Imaging.

[9]  Pierre Gouton,et al.  Spectral Characterization of a Prototype SFA Camera for Joint Visible and NIR Acquisition , 2016, Sensors.

[10]  Masatoshi Okutomi,et al.  A Practical One-Shot Multispectral Imaging System Using a Single Image Sensor , 2015, IEEE Transactions on Image Processing.

[11]  Suk Hwan Lim,et al.  Spatially Varying Color Correction (SVCC) Matrices for Reduced Noise , 2004, Color Imaging Conference.

[12]  Raja Bala,et al.  Two-dimensional transforms for device color correction and calibration , 2005, IEEE Transactions on Image Processing.

[13]  Mark S. Drew,et al.  Matrix Calculations for Digital Photography , 1997, Color Imaging Conference.

[14]  Gaurav Sharma,et al.  Figures of merit for color scanners , 1997, IEEE Trans. Image Process..

[15]  Peter A. Rhodes,et al.  A study of digital camera colorimetric characterisation based on polynomial modelling , 2001 .

[16]  Jon Y. Hardeberg,et al.  Colorimetric Characterization of Digital Cameras Preserving Hue Planes , 2005, Color Imaging Conference.

[17]  Raimondo Schettini,et al.  Color correction pipeline optimization for digital cameras , 2013, J. Electronic Imaging.

[18]  Matthew Anderson,et al.  Proposal for a Standard Default Color Space for the Internet - sRGB , 1996, CIC.

[19]  H. Joel Trussell,et al.  Color estimation under Poisson noise , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Leonardo Vanneschi,et al.  Empirical modeling for colorimetric characterization of digital cameras , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[21]  Henry R. Kang Computational Color Technology , 2006 .

[22]  Sabine Süsstrunk,et al.  What is the space of spectral sensitivity functions for digital color cameras? , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[23]  Stephen Westland,et al.  A comparative study of the characterisation of colour cameras by means of neural networks and polynomial transforms , 2004 .

[24]  Michal Mackiewicz,et al.  Color Correction Using Root-Polynomial Regression , 2015, IEEE Transactions on Image Processing.

[25]  Igor Kharitonenko,et al.  Suppression of noise amplification during colour correction , 2002, IEEE Trans. Consumer Electron..

[26]  H. Joel Trussell,et al.  Optimal color filters in the presence of noise , 1995, IEEE Trans. Image Process..

[27]  Karen O. Egiazarian,et al.  Practical Poissonian-Gaussian Noise Modeling and Fitting for Single-Image Raw-Data , 2008, IEEE Transactions on Image Processing.

[28]  Roy S. Berns,et al.  Spectral sensitivity optimization of color image sensors considering photon shot noise , 2009, J. Electronic Imaging.

[29]  H. Joel Trussell,et al.  Color scanner calibration via a neural network , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[30]  B A Wandell,et al.  Linear models of surface and illuminant spectra. , 1992, Journal of the Optical Society of America. A, Optics and image science.

[31]  Graham D. Finlayson,et al.  Colour Correction Toolbox , 2017 .

[32]  Wanqing Li,et al.  Novel color processing architecture for digital cameras with CMOS image sensors , 2005, IEEE Transactions on Consumer Electronics.

[33]  Michael J. Vrhel,et al.  Color correction using principal components , 1992 .

[34]  Brian A. Wandell,et al.  Color estimation error trade-offs , 2003, IS&T/SPIE Electronic Imaging.

[35]  Leonardo Vanneschi,et al.  Polynomial modeling and optimization for colorimetric characterization of scanners , 2008, J. Electronic Imaging.

[36]  Raimondo Schettini,et al.  Error-Tolerant Color Rendering for Digital Cameras , 2014, Journal of Mathematical Imaging and Vision.

[37]  Masatoshi Okutomi,et al.  Tunable color correction between linear and polynomial models for noisy images , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[38]  Shuxue Quan Analytical approach to the optimal linear matrix with comprehensive error metric , 2004, IS&T/SPIE Electronic Imaging.

[39]  N. Shimano Suppression of Noise Effects in Color Correction by Spectral Sensitivities of Image Sensors , 2002 .

[40]  Raimondo Schettini,et al.  A New Method for RGB to XYZ Transformation Based on Pattern Search Optimization , 2007, IEEE Transactions on Consumer Electronics.

[41]  Arcangelo Bruna,et al.  Color space transformations for digital photography exploiting information about the illuminant estimation process. , 2012, Journal of the Optical Society of America. A, Optics, image science, and vision.

[42]  R. Berns,et al.  Error propagation analysis in color measurement and imaging , 1997 .

[43]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[44]  Javier Vazquez-Corral,et al.  Perceptual Color Characterization of Cameras , 2014, Sensors.

[45]  Robert B. Fisher,et al.  Color Homography: Theory and Applications , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Brian V. Funt,et al.  Irradiance-independent camera color calibration , 2014 .

[47]  Po-Chieh Hung,et al.  Colorimetric calibration in electronic imaging devices using a look-up-table model and interpolations , 1993, J. Electronic Imaging.

[48]  Xinhao Liu,et al.  Single-Image Noise Level Estimation for Blind Denoising , 2013, IEEE Transactions on Image Processing.

[49]  Maya R. Gupta,et al.  Optimized Regression for Efficient Function Evaluation , 2012, IEEE Transactions on Image Processing.

[50]  Xuemei Zhang,et al.  Bayesian Color Correction Method for Non-Colorimetric Digital Image Sensors , 2004, Color Imaging Conference.

[51]  Cormac Herley,et al.  Trade-offs between color saturation and noise sensitivity in image sensors , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[52]  Yap-Peng Tan,et al.  Method for color correction with noise consideration , 1999, Electronic Imaging.

[53]  Masatoshi Okutomi,et al.  Pseudo four-channel image denoising for noisy CFA raw data , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[54]  Graham Finlayson,et al.  Method for hue plane preserving color correction. , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[55]  S. Westland,et al.  Color Camera Characterisation Using Artificial Neural Networks , 2002 .

[56]  Mark S. Drew,et al.  The Maximum Ignorance Assumption with Positivity , 1996, Color Imaging Conference.

[57]  Henry R. Kang,et al.  Neural network applications to the color scanner and printer calibrations , 1992, J. Electronic Imaging.

[58]  David Connah,et al.  Weighted Constrained Hue-Plane Preserving Camera Characterization , 2016, IEEE Transactions on Image Processing.

[59]  Wanqing Li,et al.  Image color correction in DCT domain , 2013, 2013 IEEE Third International Conference on Consumer Electronics ¿ Berlin (ICCE-Berlin).

[60]  Masatoshi Okutomi,et al.  Effective color correction pipeline for a noisy image , 2016, 2016 IEEE International Conference on Image Processing (ICIP).