Generative Models for Multi-Illumination Color Constancy

In this paper, the aim is multi-illumination color constancy. However, most of the existing color constancy methods are designed for single light sources. Furthermore, datasets for learning multiple illumination color constancy are largely missing. We propose a seed (physics driven) based multi-illumination color constancy method. GANs are exploited to model the illumination estimation problem as an image-to-image domain translation problem. Additionally, a novel multi-illumination data augmentation method is proposed. Experiments on single and multi-illumination datasets show that our methods outperform sota methods.

[1]  Stephen Lin,et al.  FC^4: Fully Convolutional Color Constancy with Confidence-Weighted Pooling , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Mahdi Nezamabadi,et al.  Color Appearance Models , 2014, J. Electronic Imaging.

[3]  Tom Minka,et al.  Bayesian Color Constancy with Non-Gaussian Models , 2003, NIPS.

[4]  Seoung Wug Oh,et al.  Approaching the computational color constancy as a classification problem through deep learning , 2016, Pattern Recognit..

[5]  Jung-Woo Ha,et al.  StarGAN: Unified Generative Adversarial Networks for Multi-domain Image-to-Image Translation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Jiri Matas,et al.  Recurrent Color Constancy , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[7]  Harshad Rai,et al.  Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks , 2018 .

[8]  Theo Gevers,et al.  Color Constancy for Multiple Light Sources , 2012, IEEE Transactions on Image Processing.

[9]  Michael S. Brown,et al.  Leveraging the Availability of Two Cameras for Illuminant Estimation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[11]  Shen Yan,et al.  Multiple Illumination Estimation with End-to-End Network , 2018, 2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC).

[12]  Theo Gevers,et al.  Color Constancy Using Natural Image Statistics and Scene Semantics , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Joost van de Weijer,et al.  Multi-Illuminant Estimation With Conditional Random Fields , 2014, IEEE Transactions on Image Processing.

[14]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[15]  Brian V. Funt,et al.  A Large Image Database for Color Constancy Research , 2003, CIC.

[16]  Akbar Sheikh Akbari,et al.  Color Constancy Algorithm for Mixed-Illuminant Scene Images , 2018, IEEE Access.

[17]  Mark S. Drew,et al.  Exemplar-Based Color Constancy and Multiple Illumination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Moncef Gabbouj,et al.  INTEL-TUT Dataset for Camera Invariant Color Constancy Research , 2017, ArXiv.

[19]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[20]  Theo Gevers,et al.  Color Constancy by Deep Learning , 2015, BMVC.

[21]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[22]  Jiri Matas,et al.  On Finding Gray Pixels , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Keigo Hirakawa,et al.  Color Constancy with Spatio-Spectral Statistics , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Raimondo Schettini,et al.  Color constancy using CNNs , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[25]  John K. Tsotsos,et al.  From [R, G, B] to Surface Reflectance: Computing Color Constant Descriptors in Images , 1987, IJCAI.

[26]  Oleksii Sidorov,et al.  Conditional GANs for Multi-Illuminant Color Constancy: Revolution or yet Another Approach? , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Joost van de Weijer,et al.  Improving Color Constancy by Photometric Edge Weighting , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Tae-Hyun Oh,et al.  Semantic soft segmentation , 2018, ACM Trans. Graph..

[29]  Peter V. Gehler,et al.  A Curious Problem with Using the Colour Checker Dataset for Illuminant Estimation , 2017, CIC.

[30]  Michael S. Brown,et al.  Effective learning-based illuminant estimation using simple features , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Brian V. Funt,et al.  A comparison of computational color constancy Algorithms. II. Experiments with image data , 2002, IEEE Trans. Image Process..

[32]  Kobus Barnard,et al.  Improvements to Gamut Mapping Colour Constancy Algorithms , 2000, ECCV.

[33]  Yun-Ta Tsai,et al.  Fast Fourier Color Constancy , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Raimondo Schettini,et al.  Automatic color constancy algorithm selection and combination , 2010, Pattern Recognit..

[35]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[36]  Cordelia Schmid,et al.  Using High-Level Visual Information for Color Constancy , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[37]  G. Buchsbaum.  A spatial processor model for object colour perception , 1980 .

[38]  Alexei A. Efros,et al.  Image-to-Image Translation with Conditional Adversarial Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  E. Land.  The retinex theory of color vision. , 1977, Scientific American.

[40]  Theo Gevers,et al.  Color Constancy by GANs: An Experimental Survey , 2018, ArXiv.

[41]  Sven Loncaric,et al.  Unsupervised Learning for Color Constancy , 2017, VISIGRAPP.

[42]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[43]  Brian V. Funt,et al.  Estimating Illumination Chromaticity via Support Vector Regression , 2004, Color Imaging Conference.

[44]  Raimondo Schettini,et al.  Improving Color Constancy Using Indoor–Outdoor Image Classification , 2008, IEEE Transactions on Image Processing.