A Retinal Mechanism Inspired Color Constancy Model

In this paper, we propose a novel model for the computational color constancy, inspired by the amazing ability of the human vision system (HVS) to perceive the color of objects largely constant as the light source color changes. The proposed model imitates the color processing mechanisms in the specific level of the retina, the first stage of the HVS, from the adaptation emerging in the layers of cone photoreceptors and horizontal cells (HCs) to the color-opponent mechanism and disinhibition effect of the non-classical receptive field in the layer of retinal ganglion cells (RGCs). In particular, HC modulation provides a global color correction with cone-specific lateral gain control, and the following RGCs refine the processing with iterative adaptation until all the three opponent channels reach their stable states (i.e., obtain stable outputs). Instead of explicitly estimating the scene illuminant(s), such as most existing algorithms, our model directly removes the effect of scene illuminant. Evaluations on four commonly used color constancy data sets show that the proposed model produces competitive results in comparison with the state-of-the-art methods for the scenes under either single or multiple illuminants. The results indicate that single opponency, especially the disinhibitory effect emerging in the receptive field's subunit-structured surround of RGCs, plays an important role in removing scene illuminant(s) by inherently distinguishing the spatial structures of surfaces from extensive illuminant(s).

[1]  D. Foster Color constancy , 2011, Vision Research.

[2]  Tim Gollisch,et al.  Closed-Loop Measurements of Iso-Response Stimuli Reveal Dynamic Nonlinear Stimulus Integration in the Retina , 2012, Neuron.

[3]  M. Meister,et al.  Dynamic predictive coding by the retina , 2005, Nature.

[4]  Ruixia Xu,et al.  Convex programming colour constancy with a diagonal-offset model , 2005, IEEE International Conference on Image Processing 2005.

[5]  B J Melloni,et al.  How the retina works. , 1971, American family physician.

[6]  H Ikeda,et al.  The outer disinhibitory surround of the retinal ganglion cell receptive field , 1972, The Journal of physiology.

[7]  Tim Gollisch,et al.  Eye Smarter than Scientists Believed: Neural Computations in Circuits of the Retina , 2010, Neuron.

[8]  Theo Gevers,et al.  Color Constancy for Multiple Light Sources , 2012, IEEE Transactions on Image Processing.

[9]  C. Li,et al.  Extensive integration field beyond the classical receptive field of cat's striate cortical neurons--classification and tuning properties. , 1994, Vision research.

[10]  R. Masland The Neuronal Organization of the Retina , 2012, Neuron.

[11]  Brian V. Funt,et al.  Estimating Illumination Chromaticity via Support Vector Regression , 2004, Color Imaging Conference.

[12]  Y X Zhou,et al.  Extensive disinhibitory region beyond the classical receptive field of cat retinal ganglion cells. , 1992, Vision research.

[13]  Li Chao-Yi,et al.  Extensive disinhibitory region beyond the classical receptive field of cat retinal ganglion cells , 1992, Vision Research.

[14]  E. Land Recent advances in retinex theory , 1986, Vision Research.

[15]  Yongjie Li,et al.  Efficient Color Constancy with Local Surface Reflectance Statistics , 2014, ECCV.

[16]  J. Pokorny,et al.  Horizontal cells reveal cone type-specific adaptation in primate retina. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[17]  C. Joselevitch,et al.  Human retinal circuitry and physiology , 2008 .

[18]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[19]  K. Gegenfurtner,et al.  Cortical mechanisms of colour vision , 2003, Nature Reviews Neuroscience.

[20]  Raimondo Schettini,et al.  Automatic color constancy algorithm selection and combination , 2010, Pattern Recognit..

[21]  Chin-Teng Lin,et al.  Bio-Inspired Computer Fovea Model Based on Hexagonal-Type Cellular Neural Network , 2007, IEEE Transactions on Circuits and Systems I: Regular Papers.

[22]  Leif H. Finkel,et al.  Network simulations of retinal and cortical contributions to color constancy , 1995, Vision Research.

[23]  Theo Gevers,et al.  Color Constancy using Natural Image Statistics , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Graham D. Finlayson,et al.  Corrected-Moment Illuminant Estimation , 2013, 2013 IEEE International Conference on Computer Vision.

[25]  John J. McCann,et al.  Retinex in Matlab , 2000, CIC.

[26]  Joost van de Weijer,et al.  Computational Color Constancy: Survey and Experiments , 2011, IEEE Transactions on Image Processing.

[27]  Theo Gevers,et al.  Color Constancy Using Natural Image Statistics and Scene Semantics , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[28]  Rodney M. Goodman,et al.  A real-time neural system for color constancy , 1991, IEEE Trans. Neural Networks.

[29]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[30]  Steven D. Hordley,et al.  Scene illuminant estimation: Past, present, and future , 2006 .

[31]  Joost van de Weijer,et al.  Generalized Gamut Mapping using Image Derivative Structures for Color Constancy , 2008, International Journal of Computer Vision.

[32]  Graham D. Finlayson,et al.  Color by Correlation: A Simple, Unifying Framework for Color Constancy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  C. Enroth-Cugell,et al.  The contrast sensitivity of retinal ganglion cells of the cat , 1966, The Journal of physiology.

[34]  Brian V. Funt,et al.  A data set for color research , 2002 .

[35]  K. Ikeuchi,et al.  Color constancy through inverse-intensity chromaticity space. , 2004, Journal of the Optical Society of America. A, Optics, image science, and vision.

[36]  David A. Forsyth,et al.  A novel algorithm for color constancy , 1990, International Journal of Computer Vision.

[37]  Graham D. Finlayson,et al.  Shades of Gray and Colour Constancy , 2004, CIC.

[38]  G. Buchsbaum A spatial processor model for object colour perception , 1980 .

[39]  S. Baccus Timing and computation in inner retinal circuitry. , 2007, Annual review of physiology.

[40]  Carlo Gatta,et al.  A Spatially Variant White-Patch and Gray-World Method for Color Image Enhancement Driven by Local Contrast , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  John J. McCann,et al.  Lessons Learned from Mondrians Applied to Real Images and Color Gamuts , 1999, CIC.

[42]  Hedva Spitzer,et al.  Computational adaptation model and its predictions for color induction of first and second orders , 2004, Vision Research.

[43]  J. Mollon,et al.  Human visual pigments: microspectrophotometric results from the eyes of seven persons , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[44]  Marc Ebner,et al.  Color Constancy , 2007, Computer Vision, A Reference Guide.

[45]  P. A. Dufort,et al.  Color categorization and color constancy in a neural network model of V4 , 1991, Biological Cybernetics.

[46]  C. Li,et al.  Effects of patterned backgrounds on responses of lateral geniculate neurons in cat , 2004, Experimental Brain Research.

[47]  Keigo Hirakawa,et al.  Color Constancy with Spatio-Spectral Statistics , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Ingeborg Tastl,et al.  Gamut Constrained Illuminant Estimation , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[49]  Mark S. Drew,et al.  Exemplar-Based Colour Constancy , 2012, BMVC.

[50]  Jeanny Hérault A model of colour processing in the retina of vertebrates: From photoreceptors to colour opposition and colour constancy phenomena , 1996, Neurocomputing.

[51]  Graham D. Finlayson,et al.  The correlation of reproduction and recovery angular errors for similar and diverse scenes , 2015 .

[52]  C.-Y. Li Integration Fields Beyond the Classical Receptive Field: Organization and Functional Properties , 1996 .

[53]  Y. X. Zhow,et al.  Role of the extensive area outside the X-cell receptive field in brightness information transmission. , 1991, Vision research.

[54]  Maarten Kamermans,et al.  Lateral Gain Control in the Outer Retina Leads to Potentiation of Center Responses of Retinal Neurons , 2009, The Journal of Neuroscience.

[55]  Bevil R. Conway,et al.  Advances in Color Science: From Retina to Behavior , 2010, The Journal of Neuroscience.

[56]  Leif H. Finkel,et al.  A multistage neural network for color constancy and color induction , 1995, IEEE Trans. Neural Networks.

[57]  Kobus Barnard,et al.  Practical colour constancy , 1999 .

[58]  Robby T. Tan,et al.  Color constancy through inverse-intensity chromaticity space. , 2004 .

[59]  Graham D. Finlayson,et al.  Reproduction Angular Error: An Improved Performance Metric for Illuminant Estimation , 2014, BMVC.

[60]  Hedva Spitzer,et al.  Color constancy: a biological model and its application for still and video images , 2002, Pattern Recognit..

[61]  G. Buchsbaum,et al.  Trichromacy, opponent colours coding and optimum colour information transmission in the retina , 1983, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[62]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[63]  R. W. Rodieck,et al.  Response of cat retinal ganglion cells to moving visual patterns. , 1965, Journal of neurophysiology.

[64]  Andrew Blake,et al.  Bayesian color constancy revisited , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  R. Desimone,et al.  Spectral properties of V4 neurons in the macaque , 1990, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[66]  D. Tolhurst,et al.  Concealed colour opponency in ganglion cells of the rhesus monkey retina. , 1975, The Journal of physiology.

[67]  M. Kamermans,et al.  The contribution of the outer retina to color constancy: A general model for color constancy synthesized from primate and fish data , 2007, Visual Neuroscience.

[68]  Yongjie Li,et al.  A Color Constancy Model with Double-Opponency Mechanisms , 2013, 2013 IEEE International Conference on Computer Vision.

[69]  Mark S. Drew,et al.  Exemplar-Based Color Constancy and Multiple Illumination , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[70]  R. Segev,et al.  Adaptive Colour Contrast Coding in the Salamander Retina Efficiently Matches Natural Scene Statistics , 2013, PloS one.

[71]  Bevil R Conway,et al.  A different point of hue. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[72]  Joost van de Weijer,et al.  Multi-Illuminant Estimation With Conditional Random Fields , 2014, IEEE Transactions on Image Processing.

[73]  S. D. Hordley,et al.  Reevaluation of color constancy algorithm performance. , 2006, Journal of the Optical Society of America. A, Optics, image science, and vision.

[74]  Barry B. Lee,et al.  The 'blue-on' opponent pathway in primate retina originates from a distinct bistratified ganglion cell type , 1994, Nature.

[75]  Brian V. Funt,et al.  A Large Image Database for Color Constancy Research , 2003, CIC.