Tone Mapping Beyond the Classical Receptive Field

Some neurons in the primary visual cortex (V1) of human visual system (HVS) conduct dynamic center-surround computation, which is thought to contribute to compress the high dynamic range (HDR) scene and preserve the details. We simulate this dynamic receptive field (RF) property of V1 neurons to solve the so-called tone mapping (TM) task in this paper. The novelties of our method are as follows. (1) Cortical processing mechanisms of HVS are modeled to build a local TM operation based on two Gaussian functions whose kernels and weights adapt according to the center-surround contrast, thus reducing halo artifacts and effectively enhancing the local details of bright and dark parts of image. (2) Our method uses an adaptive filter that follows the contrast levels of the image, which is computationally very efficient. (3) The local fusion between the center and surround responses returned by a cortical processing flow and the global signals returned by a sub-cortical processing flow according to the local contrast forms a dynamic mechanism that selectively enhances the details. Extensive experiments show that the proposed method can efficiently render the HDR scenes with good contrast, clear details, and high structural fidelity. In addition, the proposed method can also obtain promising performance when applied to enhance the low-light images. Furthermore, by modeling these biological solutions, our technique is simple and robust considering that our results were obtained using the same parameters for all the datasets (e.g., HDR images or low-light images), that is, mimicking how HVS operates.

[1]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[2]  Rynson W. H. Lau,et al.  Image Correction via Deep Reciprocating HDR Transformation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Greg Ward,et al.  A Contrast-Based Scalefactor for Luminance Display , 1994, Graphics Gems.

[4]  Laurence Meylan,et al.  Model of retinal local adaptation for the tone mapping of color filter array images. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[5]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[6]  A. Ardeshir Goshtasby,et al.  Fusion of multi-exposure images , 2005, Image Vis. Comput..

[7]  Hai-Miao Hu,et al.  Naturalness Preserved Enhancement Algorithm for Non-Uniform Illumination Images , 2013, IEEE Transactions on Image Processing.

[8]  Donald P. Greenberg,et al.  A multiscale model of adaptation and spatial vision for realistic image display , 1998, SIGGRAPH.

[9]  Robert L. Stevenson,et al.  Dynamic range improvement through multiple exposures , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[10]  J. Movshon,et al.  Nature and interaction of signals from the receptive field center and surround in macaque V1 neurons. , 2002, Journal of neurophysiology.

[11]  J. B. Levitt,et al.  Comparison of Spatial Summation Properties of Neurons in Macaque V1 and V2 , 2009, Journal of neurophysiology.

[12]  Hui Li,et al.  An Adaptive Method for Image Dynamic Range Adjustment , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  R. Wurtz,et al.  Activity of superior colliculus in behaving monkey. I. Visual receptive fields of single neurons. , 1972, Journal of neurophysiology.

[14]  E. Peli Contrast in complex images. , 1990, Journal of the Optical Society of America. A, Optics and image science.

[15]  Jan Kautz,et al.  Consistent tone reproduction , 2008 .

[16]  Valero Laparra,et al.  Perceptually Optimized Image Rendering , 2017, Journal of the Optical Society of America. A, Optics, image science, and vision.

[17]  Mark D. Fairchild,et al.  The HDR Photographic Survey , 2007, CIC.

[18]  Turgay Çelik,et al.  Contextual and Variational Contrast Enhancement , 2011, IEEE Transactions on Image Processing.

[19]  Donald P. Greenberg,et al.  Time-dependent visual adaptation for fast realistic image display , 2000, SIGGRAPH.

[20]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[21]  Xiao-Ping Zhang,et al.  A Probabilistic Method for Image Enhancement With Simultaneous Illumination and Reflectance Estimation , 2015, IEEE Transactions on Image Processing.

[22]  Greg Turk,et al.  LCIS: a boundary hierarchy for detail-preserving contrast reduction , 1999, SIGGRAPH.

[23]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[24]  C. R. Hoffman,et al.  Illumination and Reflection Maps : Simulated Objects in Simulated and Real Environments Gene , 1984 .

[25]  Xuelong Li,et al.  Exposure Fusion Using Boosting Laplacian Pyramid , 2014, IEEE Transactions on Cybernetics.

[26]  Zhengguo Li,et al.  Single image brightening via exposure fusion , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Yong-Jie Li,et al.  A Retina Inspired Model for High Dynamic Range Image Rendering , 2016, BICS.

[28]  Robert H. Wurtz,et al.  Signals Conveyed in the Pulvinar Pathway from Superior Colliculus to Cortical Area MT , 2011, The Journal of Neuroscience.

[29]  Xiaoyan Sun,et al.  Contrast Enhancement Based on Intrinsic Image Decomposition , 2017, IEEE Transactions on Image Processing.

[30]  Lei Zhang,et al.  Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images , 2018, IEEE Transactions on Image Processing.

[31]  M. Carandini,et al.  Functional Mechanisms Shaping Lateral Geniculate Responses to Artificial and Natural Stimuli , 2008, Neuron.

[32]  Karol Myszkowski,et al.  Adaptive Logarithmic Mapping For Displaying High Contrast Scenes , 2003, Comput. Graph. Forum.

[33]  John H. R. Maunsell,et al.  Coding of image contrast in central visual pathways of the macaque monkey , 1990, Vision Research.

[34]  A. Angelucci,et al.  Circuits and Mechanisms for Surround Modulation in Visual Cortex. , 2017, Annual review of neuroscience.

[35]  Yongjie Li,et al.  A Retinal Adaptation Model for HDR Image Compression , 2017, CCCV.

[36]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[37]  David Zhang,et al.  A Hybrid l1-l0 Layer Decomposition Model for Tone Mapping , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  Xavier Otazu,et al.  Which tone-mapping operator is the best? A comparative study of perceptual quality , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[39]  Erik Reinhard,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Dynamic Range Reduction Inspired by Photoreceptor Physiology , 2022 .

[40]  Erik Reinhard,et al.  Photographic tone reproduction for digital images , 2002, ACM Trans. Graph..

[41]  Hans-Peter Seidel,et al.  A perceptual framework for contrast processing of high dynamic range images , 2006, TAP.

[42]  Robert A. Frazor,et al.  Independence of luminance and contrast in natural scenes and in the early visual system , 2005, Nature Neuroscience.

[43]  W. Usrey,et al.  Parallel Processing in the Corticogeniculate Pathway of the Macaque Monkey , 2009, Neuron.

[44]  Boualem Boashash,et al.  Image fusion-based contrast enhancement , 2012, EURASIP Journal on Image and Video Processing.

[45]  Paul R. Martin,et al.  Extraclassical Receptive Field Properties of Parvocellular, Magnocellular, and Koniocellular Cells in the Primate Lateral Geniculate Nucleus , 2002, The Journal of Neuroscience.

[46]  Zhou Wang,et al.  Objective Quality Assessment of Tone-Mapped Images , 2013, IEEE Transactions on Image Processing.

[47]  J. R. Raol,et al.  Pixel-level Image Fusion using Wavelets and Principal Component Analysis , 2008 .

[48]  Wen Gao,et al.  Blind Quality Assessment of Tone-Mapped Images Via Analysis of Information, Naturalness, and Structure , 2016, IEEE Transactions on Multimedia.

[49]  C. Enroth-Cugell,et al.  Chapter 9 Visual adaptation and retinal gain controls , 1984 .

[50]  Rabab Kreidieh Ward,et al.  Optimizing a Tone Curve for Backward-Compatible High Dynamic Range Image and Video Compression , 2011, IEEE Transactions on Image Processing.

[51]  Jack Tumblin,et al.  The Trilateral Filter for High Contrast Images and Meshes , 2003, Rendering Techniques.

[52]  A. Oppenheim,et al.  Nonlinear filtering of multiplied and convolved signals , 1968 .

[53]  Farran Briggs,et al.  Corticogeniculate feedback and visual processing in the primate , 2011, The Journal of physiology.

[54]  D H Brainard,et al.  Analysis of the retinex theory of color vision. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[55]  Zia-ur Rahman,et al.  Properties and performance of a center/surround retinex , 1997, IEEE Trans. Image Process..

[56]  C. Gilbert,et al.  Improvement in visual sensitivity by changes in local context: Parallel studies in human observers and in V1 of alert monkeys , 1995, Neuron.

[57]  Yongjie Li,et al.  High Dynamic Range Image Rendering with a Luminance-Chromaticity Independent Model , 2015, IScIDE.

[58]  D. Pollen,et al.  Striate cortex increases contrast gain of macaque LGN neurons , 2000, Visual Neuroscience.

[59]  Sangkeun Lee,et al.  An Efficient Content-Based Image Enhancement in the Compressed Domain Using Retinex Theory , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[60]  Ming Zhang,et al.  Combining Bottom-Up and Top-Down Visual Mechanisms for Color Constancy Under Varying Illumination , 2019, IEEE Transactions on Image Processing.

[61]  D. Bradley,et al.  Structure and function of visual area MT. , 2005, Annual review of neuroscience.

[62]  Jonathan T. Barron,et al.  Deep bilateral learning for real-time image enhancement , 2017, ACM Trans. Graph..

[63]  Jessica K. Hodgins,et al.  Two methods for display of high contrast images , 1999, TOGS.

[64]  Zia-ur Rahman,et al.  A multiscale retinex for bridging the gap between color images and the human observation of scenes , 1997, IEEE Trans. Image Process..

[65]  Kate Devlin,et al.  A review of tone reproduction techniques , 2002 .

[66]  Laurence Meylan,et al.  High dynamic range image rendering with a retinex-based adaptive filter , 2006, IEEE Transactions on Image Processing.

[67]  Shiqian Wu,et al.  Weighted Guided Image Filtering , 2016, IEEE Transactions on Image Processing.

[68]  Yonghong Tian,et al.  Quality Assessment for Comparing Image Enhancement Algorithms , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[69]  Brian A. Wandell,et al.  Rendering high dynamic range images , 2000, Electronic Imaging.

[70]  Alessandra Angelucci,et al.  Beyond the Classical Receptive Field : Surround Modulation in Primary Visual Cortex , 2022 .

[71]  Kurt Debattista,et al.  Advanced High Dynamic Range Imaging: Theory and Practice , 2011 .

[72]  Barry B. Lee,et al.  Suppressive Surrounds and Contrast Gain in Magnocellular-Pathway Retinal Ganglion Cells of Macaque , 2006, The Journal of Neuroscience.

[73]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[74]  Rafal Mantiuk,et al.  Display adaptive tone mapping , 2008, ACM Trans. Graph..

[75]  Christophe Schlick,et al.  Quantization Techniques for Visualization of High Dynamic Range Pictures , 1995 .

[76]  Jian Sun,et al.  Guided Image Filtering , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[77]  K. Gegenfurtner,et al.  Cortical mechanisms of colour vision , 2003, Nature Reviews Neuroscience.

[78]  Zhengguo Li,et al.  Detail-Enhanced Multi-Scale Exposure Fusion , 2017, IEEE Transactions on Image Processing.