Non-Local Retinex - A Unifying Framework and Beyond

In this paper, we provide a short review of Retinex and then present a unifying framework. The fundamental assumption of all Retinex models is that the observed image is a multiplication between the illumination and the true underlying reflectance of the object. Starting from Morel's 2010 PDE model, where illumination is supposed to vary smoothly and where the reflectance is thus recovered from a hard-thresholded Laplacian of the observed image in a Poisson equation, we define our unifying Retinex model in two similar, but more general, steps. We reinterpret the gradient thresholding model as variational models with sparsity constraints. First, we look for a filtered gradient that is the solution of an optimization problem consisting of two terms: a sparsity prior of the reflectance and a fidelity prior of the reflectance gradient to the observed image gradient. Second, since this filtered gradient almost certainly is not a consistent image gradient, we then fit an actual reflectance gradient to it, subje...

[1]  E. Jaynes Information Theory and Statistical Mechanics , 1957 .

[2]  R. Jampel Mach Bands, Quantitative Studies on Neural Networks in the Retina. , 1966 .

[3]  E. Land,et al.  Lightness and retinex theory. , 1971, Journal of the Optical Society of America.

[4]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[5]  Berthold K. P. Horn,et al.  Determining lightness from an image , 1974, Comput. Graph. Image Process..

[6]  D Marr,et al.  The computation of lightness by the primate retina. , 1974, Vision research.

[7]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[8]  Dimitri P. Bertsekas,et al.  Constrained Optimization and Lagrange Multiplier Methods , 1982 .

[9]  E H Land,et al.  Recent advances in retinex theory and some implications for cortical computations: color vision and the natural image. , 1983, Proceedings of the National Academy of Sciences of the United States of America.

[10]  Jong-Sen Lee,et al.  Digital image smoothing and the sigma filter , 1983, Comput. Vis. Graph. Image Process..

[11]  A. Blake,et al.  On Lightness Computation in Mondrian World , 1985 .

[12]  Andrew Blake,et al.  Boundary conditions for lightness computation in Mondrian World , 1985, Comput. Vis. Graph. Image Process..

[13]  Silviu Guiasu,et al.  The principle of maximum entropy , 1985 .

[14]  E H Land,et al.  An alternative technique for the computation of the designator in the retinex theory of color vision. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[15]  A Hurlbert,et al.  Formal connections between lightness algorithms. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[16]  R. Glowinski,et al.  Augmented Lagrangian and Operator-Splitting Methods in Nonlinear Mechanics , 1987 .

[17]  Andrew Blake,et al.  Computing lightness , 1987, Pattern Recognit. Lett..

[18]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[19]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[20]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[21]  John J. McCann,et al.  Lessons Learned from Mondrians Applied to Real Images and Color Gamuts , 1999, CIC.

[22]  A. Logvinenko Lightness Induction Revisited , 1999, Perception.

[23]  John J. McCann,et al.  Retinex in Matlab , 2000, CIC.

[24]  M. Golubitsky,et al.  Geometric visual hallucinations, Euclidean symmetry and the functional architecture of striate cortex. , 2001, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[25]  D. Eagleman Visual illusions and neurobiology , 2001, Nature Reviews Neuroscience.

[26]  S. Osher,et al.  IMAGE DECOMPOSITION AND RESTORATION USING TOTAL VARIATION MINIMIZATION AND THE H−1 NORM∗ , 2002 .

[27]  Danny Barash,et al.  A Fundamental Relationship between Bilateral Filtering, Adaptive Smoothing, and the Nonlinear Diffusion Equation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Paul C. Bressloff,et al.  Visual cortex and the Retinex algorithm , 2002, IS&T/SPIE Electronic Imaging.

[29]  Mark S. Drew,et al.  Removing Shadows From Images using Retinex , 2002, CIC.

[30]  Carlo Gatta,et al.  A new algorithm for unsupervised global and local color correction , 2003, Pattern Recognit. Lett..

[31]  Stanley Osher,et al.  Image Decomposition and Restoration Using Total Variation Minimization and the H1 , 2003, Multiscale Model. Simul..

[32]  Michael Elad,et al.  Reduced complexity Retinex algorithm via the variational approach , 2003, J. Vis. Commun. Image Represent..

[33]  Michael Elad,et al.  A Variational Framework for Retinex , 2002, IS&T/SPIE Electronic Imaging.

[34]  Carlo Gatta,et al.  From Retinex to Automatic Color Equalization: issues in developing a new algorithm for unsupervised color equalization , 2004, J. Electronic Imaging.

[35]  Alfred M. Bruckstein,et al.  Diffusions and Confusions in Signal and Image Processing , 2001, Journal of Mathematical Imaging and Vision.

[36]  John J. McCann,et al.  Capturing a black cat in shade: past and present of Retinex color appearance models , 2004, J. Electronic Imaging.

[37]  Robert Sobol,et al.  Improving the Retinex algorithm for rendering wide dynamic range photographs , 2002, IS&T/SPIE Electronic Imaging.

[38]  Ron Kimmel,et al.  Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images , 2000, International Journal of Computer Vision.

[39]  Zia-ur Rahman,et al.  Retinex processing for automatic image enhancement , 2004, J. Electronic Imaging.

[40]  J. Cowan,et al.  A mathematical theory of the functional dynamics of cortical and thalamic nervous tissue , 1973, Kybernetik.

[41]  Alessandro Rizzi,et al.  Mathematical definition and analysis of the retinex algorithm. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[42]  Michael Elad,et al.  Submitted to Ieee Transactions on Image Processing Image Decomposition via the Combination of Sparse Representations and a Variational Approach , 2022 .

[43]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  Wotao Yin,et al.  Image Cartoon-Texture Decomposition and Feature Selection Using the Total Variation Regularized L1 Functional , 2005, VLSM.

[45]  Naty Hoffman Reflectance , 2006, SIGGRAPH Courses.

[46]  Cheng Lu,et al.  On the removal of shadows from images , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Dorin Comaniciu,et al.  Total variation models for variable lighting face recognition , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  M. Abidi,et al.  An Overview of Color Constancy Algorithms , 2006 .

[49]  Alessandro Rizzi,et al.  Random Spray Retinex: A New Retinex Implementation to Investigate the Local Properties of the Model , 2007, IEEE Transactions on Image Processing.

[50]  Alessandro Rizzi,et al.  Perceptual Color Correction Through Variational Techniques , 2007, IEEE Transactions on Image Processing.

[51]  Harry Shum,et al.  Natural shadow matting , 2007, TOGS.

[52]  Stephen Lin,et al.  Intrinsic image decomposition with non-local texture cues , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[53]  Robert A. Leathers,et al.  A novel method for illumination suppression in hyperspectral images , 2008, SPIE Defense + Commercial Sensing.

[54]  Carlo Gatta,et al.  A Spatially Variant White-Patch and Gray-World Method for Color Image Enhancement Driven by Local Contrast , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Guy Gilboa,et al.  Nonlocal Operators with Applications to Image Processing , 2008, Multiscale Model. Simul..

[56]  Marcelo Bertalmío,et al.  Implementing the Retinex algorithm with Wilson–Cowan equations , 2009, Journal of Physiology-Paris.

[57]  Tom Goldstein,et al.  The Split Bregman Method for L1-Regularized Problems , 2009, SIAM J. Imaging Sci..

[58]  Edoardo Provenzi,et al.  Issues About Retinex Theory and Contrast Enhancement , 2009, International Journal of Computer Vision.

[59]  Edoardo Provenzi,et al.  A Perceptually Inspired Variational Framework for Color Enhancement , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[60]  Vitomir Struc,et al.  Illumination Invariant Face Recognition by Non-Local Smoothing , 2009, COST 2101/2102 Conference.

[61]  Sylvain Paris,et al.  User-assisted intrinsic images , 2009, ACM Trans. Graph..

[62]  Akira Inoue,et al.  High dynamic range rendering for YUV images with a constraint on perceptual chroma preservation , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[63]  Guillermo Sapiro,et al.  A Variational Framework for Exemplar-Based Image Inpainting , 2011, International Journal of Computer Vision.

[64]  Jean-Michel Morel,et al.  A PDE Formalization of Retinex Theory , 2010, IEEE Transactions on Image Processing.

[65]  Petros Maragos,et al.  Tensor-based image diffusions derived from generalizations of the Total Variation and Beltrami Functionals , 2010, 2010 IEEE International Conference on Image Processing.

[66]  Jiejie Zhu,et al.  Learning to recognize shadows in monochromatic natural images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[67]  Rabab Kreidieh Ward,et al.  Compressive color imaging with group-sparsity on analysis prior , 2010, 2010 IEEE International Conference on Image Processing.

[68]  Rabab Kreidieh Ward,et al.  Non-convex group sparsity: Application to color imaging , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[69]  Anil N. Hirani,et al.  Least Squares Ranking on Graphs , 2010, 1011.1716.

[70]  Masato Tsukada,et al.  High dynamic range rendering method for YUV images with global luminance correction , 2011, 2011 IEEE International Conference on Consumer Electronics (ICCE).

[71]  Ron Kimmel,et al.  Efficient Beltrami Flow in Patch-Space , 2011, SSVM.

[72]  Alessandro Rizzi,et al.  The Art and Science of HDR Imaging: McCann/The Art and Science of HDR Imaging , 2011 .

[73]  Derek Hoiem,et al.  Single-image shadow detection and removal using paired regions , 2011, CVPR 2011.

[74]  Aichi Chien,et al.  An L1-based variational model for Retinex theory and its application to medical images , 2011, CVPR 2011.

[75]  Michael K. Ng,et al.  A Total Variation Model for Retinex , 2011, SIAM J. Imaging Sci..

[76]  Yuan Yao,et al.  Statistical ranking and combinatorial Hodge theory , 2008, Math. Program..

[77]  Wen-Chao Zhang,et al.  An improved recursive retinex for rendering high dynamic range photographs , 2011, 2011 International Conference on Wavelet Analysis and Pattern Recognition.

[78]  Chuohao Yeo,et al.  Intrinsic images decomposition using a local and global sparse representation of reflectance , 2011, CVPR 2011.

[79]  Dong-Guk Hwang,et al.  Frankle-McCann Retinex by Shuffling , 2012, ICHIT.

[80]  Stephen Lin,et al.  A Closed-Form Solution to Retinex with Nonlocal Texture Constraints , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Brian C. Lovell,et al.  Shadow detection: A survey and comparative evaluation of recent methods , 2012, Pattern Recognit..

[82]  Narendra Ahuja,et al.  Shadow Removal Using Bilateral Filtering , 2012, IEEE Transactions on Image Processing.

[83]  S. Osher,et al.  A TV Bregman iterative model of Retinex theory , 2012 .

[84]  S. Osher,et al.  Statistical ranking using the $l^{1}$-norm on graphs , 2013 .

[85]  Yung-Yu Chuang,et al.  A Non-local Sparse Model for Intrinsic Images , 2013, 2013 2nd IAPR Asian Conference on Pattern Recognition.

[86]  Stanley Osher,et al.  A unifying retinex model based on non-local differential operators , 2013, Electronic Imaging.

[87]  Derek Hoiem,et al.  Paired Regions for Shadow Detection and Removal , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[88]  Stanley Osher,et al.  Optimal data collection for informative rankings expose well-connected graphs , 2012, J. Mach. Learn. Res..

[89]  Shemels Geletaw Penalty and Augmented Lagrangian Methods to Solve Nonlinear Optimization Prob- lems , 2016 .