Recurrent Color Constancy

We introduce a novel formulation of temporal color constancy which considers multiple frames preceding the frame for which illumination is estimated. We propose an end-to-end trainable recurrent color constancy network – the RCC-Net – which exploits convolutional LSTMs and a simulated sequence to learn compositional representations in space and time. We use a standard single frame color constancy benchmark, the SFU Gray Ball Dataset, which can be adapted to a temporal setting. Extensive experiments show that the proposed method consistently outperforms single-frame state-of-the-art methods and their temporal variants.

[1]  Brian V. Funt,et al.  White Point Estimation for Uncalibrated Images , 1999, CIC.

[2]  Cordelia Schmid,et al.  Using High-Level Visual Information for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[3]  Ning Wang,et al.  Video-Based Illumination Estimation , 2011, CCIW.

[4]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[5]  Keigo Hirakawa,et al.  Color Constancy with Spatio-Spectral Statistics , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Li Fei-Fei,et al.  DenseCap: Fully Convolutional Localization Networks for Dense Captioning , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[8]  Joost van de Weijer,et al.  Computational Color Constancy: Survey and Experiments , 2011, IEEE Transactions on Image Processing.

[9]  Dani Lischinski,et al.  Illuminant Chromaticity from Image Sequences , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  Brian V. Funt,et al.  A data set for color research , 2002 .

[11]  W.E. Snyder,et al.  Color image processing pipeline , 2005, IEEE Signal Processing Magazine.

[12]  Raimondo Schettini,et al.  Color constancy using CNNs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[13]  Theo Gevers,et al.  Color Constancy Using Natural Image Statistics and Scene Semantics , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[15]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Theo Gevers,et al.  Color Constancy by Deep Learning , 2015, BMVC.

[17]  Yann LeCun,et al.  Orthogonal RNNs and Long-Memory Tasks , 2016, ArXiv.

[18]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[19]  Andrew Blake,et al.  Bayesian color constancy revisited , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Jonathan T. Barron,et al.  Convolutional Color Constancy , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[21]  Graham D. Finlayson,et al.  Colour constancy using the chromagenic constraint , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Michael S. Brown,et al.  Effective learning-based illuminant estimation using simple features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jiri Matas,et al.  Deep structured-output regression learning for computational color constancy , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[24]  Raimondo Schettini,et al.  Automatic color constancy algorithm selection and combination , 2010, Pattern Recognit..

[25]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[26]  J.-P. Renno,et al.  Application and Evaluation of Colour Constancy in Visual Surveillance , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[27]  Xiaoou Tang,et al.  Deep Specialized Network for Illuminant Estimation , 2016, ECCV.

[28]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[29]  Lin Wu,et al.  Deep Recurrent Convolutional Networks for Video-based Person Re-identification: An End-to-End Approach , 2016, ArXiv.

[30]  Brian V. Funt,et al.  A Large Image Database for Color Constancy Research , 2003, CIC.

[31]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[33]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[34]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[35]  Yun-Ta Tsai,et al.  Fast Fourier Color Constancy , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Ruigang Yang,et al.  A Uniform Framework for Estimating Illumination Chromaticity, Correspondence, and Specular Reflection , 2011, IEEE Transactions on Image Processing.

[37]  Kai-Fu Yang,et al.  Efficient illuminant estimation for color constancy using grey pixels , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Brian V. Funt,et al.  Estimating Illumination Chromaticity via Support Vector Regression , 2004, Color Imaging Conference.

[39]  Mark S. Drew,et al.  Exemplar-Based Color Constancy and Multiple Illumination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..