Attention-based Convolutional Neural Network for Computer Vision Color Constancy

Achieving color constancy is an important part of image preprocessing pipeline of contemporary digital cameras. Its goal is to eliminate the influence of the illumination color on the colors of the objects in the image scene. State-of-the-art results have been achieved with learning-based methods, especially when the deep learning approaches have been applied. Several methods that are combining local patches for global illumination estimations exist. However, in this paper, a new convolutional neural network architecture is proposed. It is trained to look for the regions, i.e., patches in the image where the most useful information about the scene illumination is contained. This is achieved with the attention mechanism stacked on top of the pretrained convolutional neural network. Additionally, the common problem of the lack of data in color constancy benchmark datasets is alleviated utilizing the stage-wise training. Experimental results show that the proposed approach achieves competitive results.

[1]  Oleksii Sidorov,et al.  Artificial Color Constancy via GoogLeNet with Angular Loss Function , 2018, Appl. Artif. Intell..

[2]  Kai-Fu Yang,et al.  Efficient illuminant estimation for color constancy using grey pixels , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Sven Loncaric,et al.  Color Rabbit: Guiding the distance of local maximums in illumination estimation , 2014, 2014 19th International Conference on Digital Signal Processing.

[4]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[5]  Sven Loncaric,et al.  Unsupervised Learning for Color Constancy , 2017, VISIGRAPP.

[6]  Sven Loncaric,et al.  Using the Random Sprays Retinex Algorithm for Global Illumination Estimation , 2013, ArXiv.

[7]  Claudio Cusano,et al.  Single and Multiple Illuminant Estimation Using Convolutional Neural Networks , 2015, IEEE Transactions on Image Processing.

[8]  Sven Loncaric,et al.  The Past and the Present of the Color Checker Dataset Misuse , 2019, 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA).

[9]  Leslie N. Smith,et al.  Cyclical Learning Rates for Training Neural Networks , 2015, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[10]  Andrew Blake,et al.  Bayesian color constancy revisited , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Alessandro Rizzi,et al.  Termite Retinex: a new implementation based on a colony of intelligent agents , 2014, J. Electronic Imaging.

[12]  Sven Loncaric,et al.  Flash and Storm: Fast and Highly Practical Tone Mapping based on Naka-Rushton Equation , 2018, VISIGRAPP.

[13]  Yun-Ta Tsai,et al.  Fast Fourier Color Constancy , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Jonathan T. Barron,et al.  Convolutional Color Constancy , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Lou,et al.  UvA-DARE (Digital Academic Repository) Color Constancy by Deep Learning Color Constancy by Deep Learning , 2015 .

[16]  Michael S. Brown,et al.  Effective learning-based illuminant estimation using simple features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[18]  Dilip K Prasad,et al.  Illuminant estimation for color constancy: why spatial-domain methods work and the role of the color distribution. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[19]  G. Buchsbaum A spatial processor model for object colour perception , 1980 .

[20]  Sven Loncaric,et al.  A Perceptual Measure of Illumination Estimation Error , 2015, VISAPP.

[21]  Graham D. Finlayson,et al.  Re-evaluating colour constancy algorithms , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[22]  Stephen Lin,et al.  FC^4: Fully Convolutional Color Constancy with Confidence-Weighted Pooling , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  C. Alejandro Parraga,et al.  Colour Constancy Beyond the Classical Receptive Field , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Raimondo Schettini,et al.  Color constancy using CNNs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  Theo Gevers,et al.  Color Constancy using Natural Image Statistics , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Sven Loncaric,et al.  Color Cat: Remembering Colors for Illumination Estimation , 2015, IEEE Signal Processing Letters.

[29]  Sven Loncaric,et al.  Improving the white patch method by subsampling , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[30]  Brian V. Funt,et al.  A comparison of computational color constancy algorithms. I: Methodology and experiments with synthesized data , 2002, IEEE Trans. Image Process..

[31]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[32]  Kai-Fu Yang,et al.  Color Constancy Using Double-Opponency , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Jiri Matas,et al.  Revisiting Gray Pixel for Statistical Illumination Estimation , 2019, VISIGRAPP.

[34]  Kobus Barnard,et al.  Estimating the scene illumination chromaticity by using a neural network. , 2002, Journal of the Optical Society of America. A, Optics, image science, and vision.

[35]  Cordelia Schmid,et al.  Using High-Level Visual Information for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[36]  Keigo Hirakawa,et al.  Color Constancy with Spatio-Spectral Statistics , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Sven Loncaric,et al.  Color Beaver: Bounding Illumination Estimations for Higher Accuracy , 2019, VISIGRAPP.

[38]  Sung-Min Woo,et al.  Improving Color Constancy in an Ambient Light Environment Using the Phong Reflection Model , 2018, IEEE Transactions on Image Processing.

[39]  Graham D. Finlayson,et al.  Corrected-Moment Illuminant Estimation , 2013, 2013 IEEE International Conference on Computer Vision.

[40]  Kobus Barnard,et al.  Improvements to Gamut Mapping Colour Constancy Algorithms , 2000, ECCV.

[41]  Sven Loncaric,et al.  Color Dog - Guiding the Global Illumination Estimation to Better Accuracy , 2015, VISAPP.

[42]  Lilong Shi,et al.  The Rehabilitation of MaxRGB , 2010, CIC.

[43]  Mark S. Drew,et al.  The Role of Bright Pixels in Illumination Estimation , 2012, Color Imaging Conference.

[44]  Ingeborg Tastl,et al.  Gamut Constrained Illuminant Estimation , 2006, International Journal of Computer Vision.

[45]  Sven Loncaric,et al.  Using the red chromaticity for illumination estimation , 2015, 2015 9th International Symposium on Image and Signal Processing and Analysis (ISPA).

[46]  Sven Loncaric,et al.  Blue Shift Assumption: Improving Illumination Estimation Accuracy for Single Image from Unknown Source , 2019, VISIGRAPP.

[47]  Xiaoou Tang,et al.  Deep Specialized Network for Illuminant Estimation , 2016, ECCV.

[48]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.