Real-time noise-aware tone mapping

Real-time high quality video tone mapping is needed for many applications, such as digital viewfinders in cameras, display algorithms which adapt to ambient light, in-camera processing, rendering engines for video games and video post-processing. We propose a viable solution for these applications by designing a video tone-mapping operator that controls the visibility of the noise, adapts to display and viewing environment, minimizes contrast distortions, preserves or enhances image details, and can be run in real-time on an incoming sequence without any preprocessing. To our knowledge, no existing solution offers all these features. Our novel contributions are: a fast procedure for computing local display-adaptive tone-curves which minimize contrast distortions, a fast method for detail enhancement free from ringing artifacts, and an integrated video tone-mapping solution combining all the above features.

[1]  Dominique Thoreau,et al.  Zonal brightness coherency for video tone mapping , 2014, Signal Process. Image Commun..

[2]  Guillermo Sapiro,et al.  Robust anisotropic diffusion , 1998, IEEE Trans. Image Process..

[3]  Paolo Cignoni,et al.  A Low‐Memory, Straightforward and Fast Bilateral Filter Through Subsampling in Spatial Domain , 2012, Comput. Graph. Forum.

[4]  Leonard McMillan,et al.  Video enhancement using per-pixel virtual exposures , 2005, ACM Trans. Graph..

[5]  Marc Levoy,et al.  Gaussian KD-trees for fast high-dimensional filtering , 2009, ACM Trans. Graph..

[6]  Steve Marschner,et al.  Perceptually based tone mapping of high dynamic range image streams , 2005, EGSR '05.

[7]  Jongmin Baek,et al.  Accelerating spatially varying Gaussian filters , 2010, ACM Trans. Graph..

[8]  Danny Barash,et al.  A Fundamental Relationship between Bilateral Filtering, Adaptive Smoothing, and the Nonlinear Diffusion Equation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Peyman Milanfar,et al.  Kernel Regression for Image Processing and Reconstruction , 2007, IEEE Transactions on Image Processing.

[10]  Christine D. Piatko,et al.  A visibility matching tone reproduction operator for high dynamic range scenes , 1997, SIGGRAPH '97.

[11]  Rabab Kreidieh Ward,et al.  Optimizing a Tone Curve for Backward-Compatible High Dynamic Range Image and Video Compression , 2011, IEEE Transactions on Image Processing.

[12]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Sylvain Paris,et al.  Real-time edge-aware image processing with the bilateral grid , 2007, ACM Trans. Graph..

[14]  F. Durand,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002, ACM Trans. Graph..

[15]  Peyman Milanfar,et al.  A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical , 2013, IEEE Signal Processing Magazine.

[16]  Alexander Belyaev,et al.  Fast Gauss Bilateral Filtering , 2010, Comput. Graph. Forum.

[17]  Stefan Gustavson,et al.  Unified HDR reconstruction from raw CFA data , 2013, IEEE International Conference on Computational Photography (ICCP).

[18]  Roy S. Berns,et al.  Methods for characterizing CRT displays , 1996 .

[19]  Erik Reinhard,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Dynamic Range Reduction Inspired by Photoreceptor Physiology , 2022 .

[20]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[21]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[22]  Jörg Weule,et al.  Non-Linear Gaussian Filters Performing Edge Preserving Diffusion , 1995, DAGM-Symposium.

[23]  Andrew Adams,et al.  Fast High‐Dimensional Filtering Using the Permutohedral Lattice , 2010, Comput. Graph. Forum.

[24]  Luís Paulo Santos,et al.  A local model of eye adaptation for high dynamic range images , 2004, AFRIGRAPH '04.

[25]  Karen O. Egiazarian,et al.  Practical Poissonian-Gaussian Noise Modeling and Fitting for Single-Image Raw-Data , 2008, IEEE Transactions on Image Processing.

[26]  J. H. van Hateren,et al.  Encoding of high dynamic range video with a model of human cones , 2006, TOGS.

[27]  Aljoscha Smolic,et al.  Suplemental Material for Temporally Coherent Local Tone Mapping of HDR Video , 2014 .

[28]  Harald Brendel,et al.  Creating cinematic wide gamut HDR-video for the evaluation of tone mapping operators and HDR-displays , 2014, Electronic Imaging.

[29]  Qingxiong Yang,et al.  Recursive Bilateral Filtering , 2012, ECCV.

[30]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, ACM Trans. Graph..

[31]  Sarah R. Allred,et al.  Lightness perception in high dynamic range images: local and remote luminance effects. , 2012, Journal of vision.

[32]  Rafal Mantiuk,et al.  Display adaptive tone mapping , 2008, ACM Trans. Graph..

[33]  Adrien Gruson,et al.  Temporal coherency for video tone mapping , 2012, Other Conferences.

[34]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[35]  Frédo Durand,et al.  Fast Local Laplacian Filters , 2014, ACM Trans. Graph..

[36]  Erik Reinhard,et al.  Real Time Automated Tone Mapping System for HDR Video , 2012, ICIP 2012.

[37]  Robert Wanat,et al.  Evaluation of Tone Mapping Operators for HDR-Video , 2013, Comput. Graph. Forum.

[38]  Wolfgang Heidrich,et al.  Color correction for tone mapping , 2009, Comput. Graph. Forum.

[39]  Peyman Milanfar,et al.  Higher order bilateral filters and their properties , 2007, Electronic Imaging.

[40]  Jaakko Astola,et al.  Local Approximation Techniques in Signal and Image Processing (SPIE Press Monograph Vol. PM157) , 2006 .

[41]  Karol Myszkowski,et al.  Adaptive Logarithmic Mapping For Displaying High Contrast Scenes , 2003, Comput. Graph. Forum.

[42]  Donald P. Greenberg,et al.  Time-dependent visual adaptation for fast realistic image display , 2000, SIGGRAPH.

[43]  Karen O. Egiazarian,et al.  Video Denoising, Deblocking, and Enhancement Through Separable 4-D Nonlocal Spatiotemporal Transforms , 2012, IEEE Transactions on Image Processing.

[44]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[45]  Jack Tumblin,et al.  The Trilateral Filter for High Contrast Images and Meshes , 2003, Rendering Techniques.