Fast Fourier Color Constancy

We present Fast Fourier Color Constancy (FFCC), a color constancy algorithm which solves illuminant estimation by reducing it to a spatial localization task on a torus. By operating in the frequency domain, FFCC produces lower error rates than the previous state-of-the-art by 13–20% while being 250-3000 times faster. This unconventional approach introduces challenges regarding aliasing, directional statistics, and preconditioning, which we address. By producing a complete posterior distribution over illuminants instead of a single illuminant estimate, FFCC enables better training techniques, an effective temporal smoothing technique, and richer methods for error analysis. Our implementation of FFCC runs at ~700 frames per second on a mobile device, allowing it to be used as an accurate, real-time, temporally-coherent automatic white balance algorithm.

[1]  Joost van de Weijer,et al.  Computational Color Constancy: Survey and Experiments , 2011, IEEE Transactions on Image Processing.

[2]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[3]  G D Finlayson,et al.  Color constancy at a pixel. , 2001, Journal of the Optical Society of America. A, Optics, image science, and vision.

[4]  Cordelia Schmid,et al.  Using High-Level Visual Information for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[5]  Dilip K Prasad,et al.  Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[6]  Kanti V. Mardia,et al.  Statistics of Directional Data , 1972 .

[7]  Raimondo Schettini,et al.  Automatic color constancy algorithm selection and combination , 2010, Pattern Recognit..

[8]  Raimondo Schettini,et al.  Color constancy using CNNs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[9]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[10]  Keigo Hirakawa,et al.  Color Constancy with Spatio-Spectral Statistics , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  D. Foster.  Color constancy , 2011, Vision Research.

[12]  Sven Loncaric,et al.  Color Dog - Guiding the Global Illumination Estimation to Better Accuracy , 2015, VISAPP.

[13]  Jonathan T. Barron,et al.  Convolutional Color Constancy , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[14]  Frédo Durand,et al.  Decoupling algorithms from schedules for easy optimization of image processing pipelines , 2012, ACM Trans. Graph..

[15]  D H Brainard,et al.  Analysis of the retinex theory of color vision. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[16]  Michael S. Brown,et al.  Effective learning-based illuminant estimation using simple features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[18]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[19]  References , 1971 .

[20]  Marc Ebner,et al.  Color Constancy , 2007, Computer Vision, A Reference Guide.

[21]  Mark S. Drew,et al.  Exemplar-Based Color Constancy and Multiple Illumination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Yang Song,et al.  Learning Fine-Grained Image Similarity with Deep Ranking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[23]  Xiaoou Tang,et al.  Deep Specialized Network for Illuminant Estimation , 2016, ECCV.

[24]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[25]  Graham D. Finlayson,et al.  Corrected-Moment Illuminant Estimation , 2013, 2013 IEEE International Conference on Computer Vision.

[26]  Jonathan T. Barron,et al.  Burst photography for high dynamic range and low-light imaging on mobile cameras , 2016, ACM Trans. Graph..

[27]  Kanti V. Mardia,et al.  Bayesian Methods in Structural Bioinformatics , 2012 .

[28]  Brian V. Funt,et al.  Estimating Illumination Chromaticity via Support Vector Regression , 2004, Color Imaging Conference.

[29]  Andrew Blake,et al.  Bayesian color constancy revisited , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  K. Mardia.  Statistics of Directional Data , 1972 .

[31]  Ayan Chakrabarti,et al.  Color Constancy by Learning to Predict Chromaticity from Luminance , 2015, NIPS.

[32]  Yongjie Li,et al.  Efficient Color Constancy with Local Surface Reflectance Statistics , 2014, ECCV.

[33]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[34]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[35]  Kai-Fu Yang,et al.  Efficient illuminant estimation for color constancy using grey pixels , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  H. Barrow,et al.  RECOVERING INTRINSIC SCENE CHARACTERISTICS FROM IMAGES , 1978 .

[37]  Edward H. Adelson,et al.  The perception of shading and reflectance , 1996 .

[38]  Theo Gevers,et al.  Color Constancy Using Natural Image Statistics and Scene Semantics , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.