Variational Models for Color Image Correction Inspired by Visual Perception and Neuroscience

Reproducing the perception of a real-world scene on a display device is a very challenging task which requires the understanding of the camera processing pipeline, the display process, and the way the human visual system processes the light it captures. Mathematical models based on psychophysical and physiological laws on color vision, named Retinex, provide efficient tools to handle degradations produced during the camera processing pipeline like the reduction of the contrast. In particular, Batard and Bertalmío (in J Math Imaging Vis 60(6):849–881, 2018) described some psychophysical laws on brightness perception as covariant derivatives, included them into a variational model, and observed that the quality of the color image correction is correlated with the accuracy of the vision model it includes. Based on this observation, we postulate that this model can be improved by including more accurate data on vision with a special attention on visual neuroscience here. Then, inspired by the presence of neurons responding to different visual attributes in the area V1 of the visual cortex as orientation, color or movement, to name a few, and horizontal connections modeling the interactions between those neurons, we construct two variational models to process both local (edges, textures) and global (contrast) features. This is an improvement with respect to the model of Batard and Bertalmío as the latter cannot process local and global features independently and simultaneously. Finally, we conduct experiments on color images which corroborate the improvement provided by the new models.

[1]  Mila Nikolova,et al.  Fast Hue and Range Preserving Histogram Specification: Theory and New Algorithms for Color Image Enhancement , 2014, IEEE Transactions on Image Processing.

[2]  M. Fairchild,et al.  Predicting the lightness of chromatic object colors using CIELAB , 1991 .

[3]  Vito Di Maio,et al.  Brain and vision , 2008, Brain Research.

[4]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[5]  Olivier P. Faugeras,et al.  Hyperbolic Planforms in Relation to Visual Edges and Textures Perception , 2009, PLoS Comput. Biol..

[6]  Aurélie Bugeau,et al.  Luminance-Hue Specification in the RGB Space , 2015, SSVM.

[7]  Dang-Khoa Nguyen,et al.  A Proximal Minimization Algorithm for Structured Nonconvex and Nonsmooth Problems , 2018, SIAM J. Optim..

[8]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[9]  Edoardo Provenzi,et al.  An Analysis of Visual Adaptation and Contrast Perception for Tone Mapping , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Alessandro Rizzi,et al.  Mathematical definition and analysis of the retinex algorithm. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[11]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[12]  Todor Georgiev,et al.  RELIGHTING, RETINEX THEORY, AND PERCEIVED GRADIENTS , 2005 .

[13]  Pascal Getreuer,et al.  Automatic Color Enhancement (ACE) and its Fast Implementation , 2012, Image Process. Line.

[14]  Gabriele Steidl,et al.  A Connection Between Image Processing and Artificial Neural Networks Layers Through a Geometric Model of Visual Perception , 2019, SSVM.

[15]  Marcelo Bertalmío,et al.  A Geometric Model of Brightness Perception and Its Application to Color Images Correction , 2018, Journal of Mathematical Imaging and Vision.

[16]  Edoardo Provenzi,et al.  Issues About Retinex Theory and Contrast Enhancement , 2009, International Journal of Computer Vision.

[17]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[18]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[19]  Marc Teboulle,et al.  Proximal alternating linearized minimization for nonconvex and nonsmooth problems , 2013, Mathematical Programming.

[20]  T. P. Dinh,et al.  Convex analysis approach to d.c. programming: Theory, Algorithm and Applications , 1997 .

[21]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[22]  Adam Powell,et al.  The Origins of Lactase Persistence in Europe , 2009, PLoS Comput. Biol..

[23]  Paul C. Bressloff,et al.  An Amplitude Equation Approach to Contextual Effects in Visual Cortex , 2002, Neural Computation.

[24]  Gabriele Steidl,et al.  Variational Contrast Enhancement of Gray-Scale and RGB Images , 2016, Journal of Mathematical Imaging and Vision.

[25]  R. Shapley,et al.  The Orientation Selectivity of Color-Responsive Neurons in Macaque V1 , 2008, The Journal of Neuroscience.

[26]  Alessandro Rizzi,et al.  Perceptual Color Correction Through Variational Techniques , 2007, IEEE Transactions on Image Processing.

[27]  J. Toland A duality principle for non-convex optimisation and the calculus of variations , 1979 .

[28]  Thomas Pock,et al.  Inertial Proximal Alternating Linearized Minimization (iPALM) for Nonconvex and Nonsmooth Problems , 2016, SIAM J. Imaging Sci..

[29]  Antonin Chambolle,et al.  An introduction to continuous optimization for imaging , 2016, Acta Numerica.

[30]  Marcelo Bertalmío,et al.  Image Processing for Cinema , 2014 .

[31]  Sebastian Banert,et al.  A general double-proximal gradient algorithm for d.c. programming , 2016, Mathematical Programming.

[32]  D. Jameson,et al.  An opponent-process theory of color vision. , 1957, Psychological review.

[33]  Benar Fux Svaiter,et al.  Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods , 2013, Math. Program..

[34]  M. Bertalmío,et al.  A class of nonlocal variational problems on a vector bundle for color image local contrast reduction/enhancement , 2015 .

[35]  Paul C. Bressloff,et al.  Visual cortex and the Retinex algorithm , 2002, IS&T/SPIE Electronic Imaging.

[36]  Marcelo Bertalmío,et al.  Analysis of retinal and cortical components of Retinex algorithms , 2017, J. Electronic Imaging.

[37]  Thomas Batard,et al.  A Class of Generalized Laplacians on Vector Bundles Devoted to Multi-Channel Image Processing , 2014, Journal of Mathematical Imaging and Vision.

[38]  Marcelo Bertalmío,et al.  A Nonlocal Variational Formulation for the Improvement of Tone Mapped Images , 2014, SIAM J. Imaging Sci..

[39]  Marcelo Bertalmío,et al.  Implementing the Retinex algorithm with Wilson–Cowan equations , 2009, Journal of Physiology-Paris.

[40]  Anna Song,et al.  A neural field model for color perception unifying assimilation and contrast , 2018, PLoS Comput. Biol..

[41]  S. Sutherland Eye, brain and vision , 1993, Nature.