Automated parameter tuning for tone mapping using visual saliency

While many tone mapping operators have been proposed, tuning their parameters remains a challenging task. In this work, we present a parameter tuning algorithm that can optimize the parameters of tone mapping operators automatically by minimizing the distortion in visual saliency caused by the process of tone mapping. The algorithm employs an improved saliency detection model for HDR images. Saliency distortion is quantified as the Kullback-Leibler divergence between the saliency distributions of the tone mapped images and those of the corresponding HDR images. We show that the minimization can be accomplished by employing an evolution strategy. Experiments using several tone mapping operators and a number of HDR images demonstrate the effectiveness of our algorithm. Statistical analyses are conducted to assess the improvement over default parameter settings and prior work. Graphical abstractDisplay Omitted HighlightsWe provide a universal solution for parameter tuning of tone mapping operators without the requirement for user interaction.We use the bottom-up visual saliency calculated from computational visual attention systems to measure image quality for parameter tuning.We employ an evolution strategy to solve the automatic parameter tuning as an optimization problem.Our method can improve the image quality of tone mapped images by generating results with more faithful appearance.

[1]  Hans-Peter Seidel,et al.  Predicting visible differences in high dynamic range images: model and its calibration , 2005, IS&T/SPIE Electronic Imaging.

[2]  Blain Brown,et al.  Cinematography : theory and practice : imagemaking for cinematographers and directors , 2016 .

[3]  Dirk V. Arnold,et al.  Saliency-based parameter tuning for tone mapping , 2014, CVMP.

[4]  Greg Ward,et al.  High dynamic range imaging , 2001, SIGGRAPH '04.

[5]  Patrick Le Callet,et al.  Tone mapping based HDR compression: Does it affect visual experience? , 2014, Signal Process. Image Commun..

[6]  Jan Kautz,et al.  Exposure Fusion , 2007 .

[7]  Christophe Schlick,et al.  Quantization Techniques for Visualization of High Dynamic Range Pictures , 1995 .

[8]  J ValdésJulio,et al.  2006 Special issue , 2006 .

[9]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[10]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[11]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[12]  Kenneth Chiu,et al.  Spatially Nonuniform Scaling Functions for High Contrast Images , 1993 .

[13]  Hans-Peter Seidel,et al.  Analysis of Reproducing Real‐World Appearance on Displays of Varying Dynamic Range , 2006, Comput. Graph. Forum.

[14]  Karol Myszkowski,et al.  Adaptive Logarithmic Mapping For Displaying High Contrast Scenes , 2003, Comput. Graph. Forum.

[15]  Laurent Itti,et al.  Automatic foveation for video compression using a neurobiological model of visual attention , 2004, IEEE Transactions on Image Processing.

[16]  Greg Ward,et al.  A Contrast-Based Scalefactor for Luminance Display , 1994, Graphics Gems.

[17]  Jean-Philippe Tarel,et al.  Saliency Maps of High Dynamic Range Images , 2010, ECCV Workshops.

[18]  Kurt Debattista,et al.  Advanced High Dynamic Range Imaging: Theory and Practice , 2011 .

[19]  Mark D. Fairchild,et al.  The HDR Photographic Survey , 2007, CIC.

[20]  Kai Zeng,et al.  High dynamic range image tone mapping by optimizing tone mapped image quality index , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[21]  Wen-Chieh Lin,et al.  Attention-based high dynamic range imaging , 2011, The Visual Computer.

[22]  Hans-Peter Seidel,et al.  Perceptual evaluation of tone mapping operators with real-world scenes , 2005, IS&T/SPIE Electronic Imaging.

[23]  Dirk V. Arnold,et al.  Virtual photograph based saliency analysis of high dynamic range images , 2013, CAE '13.

[24]  Zeev Farbman,et al.  Interactive local adjustment of tonal values , 2006, SIGGRAPH 2006.

[25]  R. Quentin Grafton,et al.  Mean Squared Error , 2017, Encyclopedia of Machine Learning and Data Mining.

[26]  Christof Koch,et al.  Modeling attention to salient proto-objects , 2006, Neural Networks.

[27]  Erik Reinhard,et al.  Ieee Transactions on Visualization and Computer Graphics 1 Dynamic Range Reduction Inspired by Photoreceptor Physiology , 2022 .

[28]  Taku Komura,et al.  Topology matching for fully automatic similarity estimation of 3D shapes , 2001, SIGGRAPH.

[29]  Jan Kautz,et al.  Exposure Fusion , 2009, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[30]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[31]  Erik Reinhard,et al.  Parameter Estimation for Photographic Tone Reproduction , 2002, J. Graphics, GPU, & Game Tools.

[32]  Michael Ashikhmin,et al.  A Tone Mapping Algorithm for High Contrast Images , 2002, Rendering Techniques.

[33]  Ingo Rechenberg,et al.  Evolutionsstrategie : Optimierung technischer Systeme nach Prinzipien der biologischen Evolution , 1973 .

[34]  Michael Wimmer,et al.  Image Attributes and Quality for Evaluation of Tone Mapping Operators , 2006 .

[35]  Hiroshi Yamaguchi,et al.  Testing HDR Image Rendering Algorithms , 2004, CIC.

[36]  Xin Yao,et al.  Fast Evolution Strategies , 1997, Evolutionary Programming.

[37]  Rafal Mantiuk,et al.  Display adaptive tone mapping , 2008, SIGGRAPH 2008.

[38]  Hans-Peter Seidel,et al.  Dynamic range independent image quality assessment , 2008, ACM Transactions on Graphics.

[39]  Ingrid Heynderickx,et al.  Visual Attention in Objective Image Quality Assessment: Based on Eye-Tracking Data , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Dirk V. Arnold,et al.  Tone mapping by interactive evolution , 2009, GECCO.

[41]  Michael Ashikhmin,et al.  A reality check for tone-mapping operators , 2006, TAP.

[42]  Alan Chalmers,et al.  Evaluation of tone mapping operators using a High Dynamic Range display , 2005, SIGGRAPH 2005.

[43]  S. Avidan,et al.  Seam carving for content-aware image resizing , 2007, SIGGRAPH 2007.

[44]  Mark D. Fairchild,et al.  Meet iCAM: A Next-Generation Color Appearance Model , 2002, Color Imaging Conference.

[45]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[46]  W. Vent,et al.  Rechenberg, Ingo, Evolutionsstrategie — Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. 170 S. mit 36 Abb. Frommann‐Holzboog‐Verlag. Stuttgart 1973. Broschiert , 1975 .

[47]  Henrik I. Christensen,et al.  Computational visual attention systems and their cognitive foundations: A survey , 2010, TAP.

[48]  Alan C. Bovik,et al.  Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures , 2009, IEEE Signal Processing Magazine.

[49]  Marcus Barkowsky,et al.  On the perceptual similarity of realistic looking tone mapped High Dynamic Range images , 2010, 2010 IEEE International Conference on Image Processing.

[50]  Zhou Wang,et al.  Objective Quality Assessment of Tone-Mapped Images , 2013, IEEE Transactions on Image Processing.

[51]  Christian Bloch,et al.  Style-based tone mapping for HDR images , 2013, SA '13.

[52]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.