Temporally Consistent Tone Mapping of Images and Video Using Optimal K-means Clustering

The field of high dynamic range imaging addresses the problem of capturing and displaying the large range of luminance levels found in the world, using devices with limited dynamic range. In this paper we present a novel tone mapping algorithm that is based on K-means clustering. Using dynamic programming we are able to not only solve the clustering problem efficiently, but also find the global optimum. Our algorithm runs in $$\hbox {O}(N^2K)$$O(N2K) for an image with N input luminance levels and K output levels. We show that our algorithm gives comparable results to state-of-the-art tone mapping algorithms, but with the additional large benefit of a minimum of parameters. We show how to extend the method to handle video input. We test our algorithm on a number of standard high dynamic range images and video sequences and give qualitative and quantitative comparisons to a number of state-of-the-art tone mapping algorithms.

[1]  L. McMillan,et al.  Video enhancement using per-pixel virtual exposures , 2005, SIGGRAPH 2005.

[2]  Andreas Schilling,et al.  Creating cinematic wide gamut HDR-video for the evaluation of tone mapping operators and HDR-displays , 2014 .

[3]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[4]  Éva Tardos,et al.  Algorithm design , 2005 .

[5]  Richard Szeliski,et al.  High dynamic range video , 2003, ACM Trans. Graph..

[6]  Haizhou Wang,et al.  Ckmeans.1d.dp: Optimal k-means Clustering in One Dimension by Dynamic Programming , 2011, R J..

[7]  Robert L. Stevenson,et al.  Dynamic range improvement through multiple exposures , 1999, Proceedings 1999 International Conference on Image Processing (Cat. 99CH36348).

[8]  Anders Ynnerman,et al.  A unified framework for multi-sensor HDR video reconstruction , 2013, Signal Process. Image Commun..

[9]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[10]  Wolfgang Heidrich,et al.  Color correction for tone mapping , 2009, Comput. Graph. Forum.

[11]  Paul Scheunders,et al.  A comparison of clustering algorithms applied to color image quantization , 1997, Pattern Recognit. Lett..

[12]  Robert Tibshirani,et al.  Estimating the number of clusters in a data set via the gap statistic , 2000 .

[13]  Rae-Hong Park,et al.  Local tone mapping using K-means algorithm and automatic gamma setting , 2011, 2011 IEEE International Conference on Consumer Electronics (ICCE).

[14]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics) , 2005 .

[15]  Magnus Oskarsson Democratic Tone Mapping Using Optimal K-means Clustering , 2015, SCIA.

[16]  Hans-Peter Seidel,et al.  A perceptual framework for contrast processing of high dynamic range images , 2006, TAP.

[17]  Paul Scheunders,et al.  A genetic c-Means clustering algorithm applied to color image quantization , 1997, Pattern Recognit..

[18]  J. Fleiss Measuring nominal scale agreement among many raters. , 1971 .

[19]  Charles Hansen,et al.  Adaptive temporal tone mapping , 2004 .

[20]  Shree K. Nayar,et al.  High Dynamic Range from Multiple Images: Which Exposures to Combine?∗ , 2003 .

[21]  Dominique Thoreau,et al.  Zonal brightness coherency for video tone mapping , 2014, Signal Process. Image Commun..

[22]  Peter J. Bickel,et al.  The Earth Mover's distance is the Mallows distance: some insights from statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[23]  Mehmet Celenk,et al.  A color clustering technique for image segmentation , 1990, Comput. Vis. Graph. Image Process..

[24]  Erik Reinhard,et al.  Real Time Automated Tone Mapping System for HDR Video , 2012, ICIP 2012.

[25]  Erik Reinhard,et al.  Photographic tone reproduction for digital images , 2002, ACM Trans. Graph..

[26]  Werner Purgathofer,et al.  Tone Reproduction and Physically Based Spectral Rendering , 2002, Eurographics.

[27]  Karol Myszkowski,et al.  Adaptive Logarithmic Mapping For Displaying High Contrast Scenes , 2003, Comput. Graph. Forum.

[28]  Douglas Steinley,et al.  K-means clustering: a half-century synthesis. , 2006, The British journal of mathematical and statistical psychology.

[29]  Stefan Gustavson,et al.  Unified HDR reconstruction from raw CFA data , 2013, IEEE International Conference on Computational Photography (ICCP).

[30]  J. H. van Hateren,et al.  Encoding of high dynamic range video with a model of human cones , 2006, TOGS.

[31]  Calle Lejdfors,et al.  Adaptive enhancement and noise reduction in very low light-level video , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[32]  Rafal Mantiuk,et al.  Assessment of video tone-mapping: Are cameras' S-shaped tone-curves good enough? , 2013, J. Vis. Commun. Image Represent..

[33]  Aljoscha Smolic,et al.  Suplemental Material for Temporally Coherent Local Tone Mapping of HDR Video , 2014 .

[34]  Rafal Mantiuk,et al.  Display adaptive tone mapping , 2008, SIGGRAPH 2008.

[35]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[36]  Meena Mahajan,et al.  The Planar k-means Problem is NP-hard I , 2009 .

[37]  Greg Turk,et al.  LCIS: a boundary hierarchy for detail-preserving contrast reduction , 1999, SIGGRAPH.

[38]  Christine D. Piatko,et al.  A Visibility Matching Tone Reproduction Operator for High Dynamic Range Scenes , 1997, IEEE Trans. Vis. Comput. Graph..

[39]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[40]  Steve Marschner,et al.  Perceptually based tone mapping of high dynamic range image streams , 2005, EGSR '05.

[41]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[42]  Christophe Schlick,et al.  Quantization Techniques for Visualization of High Dynamic Range Pictures , 1995 .

[43]  Donald P. Greenberg,et al.  A model of visual adaptation for realistic image synthesis , 1996, SIGGRAPH.

[44]  Patrick Le Callet,et al.  Spatio-temporal Tone Mapping Operator Based on a Retina Model , 2009, CCIW.

[45]  Sanjoy Dasgupta,et al.  Random projection trees for vector quantization , 2008, 2008 46th Annual Allerton Conference on Communication, Control, and Computing.

[46]  Christine D. Piatko,et al.  A visibility matching tone reproduction operator for high dynamic range scenes , 1997, SIGGRAPH '97.

[47]  Richard Bellman,et al.  A note on cluster analysis and dynamic programming , 1973 .

[48]  Luís Paulo Santos,et al.  A local model of eye adaptation for high dynamic range images , 2004, AFRIGRAPH '04.

[49]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[50]  Robert Wanat,et al.  Evaluation of Tone Mapping Operators for HDR-Video , 2013, Comput. Graph. Forum.

[51]  Holly E. Rushmeier,et al.  Tone reproduction for realistic images , 1993, IEEE Computer Graphics and Applications.

[52]  Rafal Mantiuk,et al.  Real-time noise-aware tone mapping , 2015, ACM Trans. Graph..

[53]  Greg Ward,et al.  A Contrast-Based Scalefactor for Luminance Display , 1994, Graphics Gems.

[54]  Donald P. Greenberg,et al.  Time-dependent visual adaptation for fast realistic image display , 2000, SIGGRAPH.

[55]  Magnus Oskarsson,et al.  The Remarkable Visual Abilities of Nocturnal Insects: Neural Principles and Bioinspired Night-Vision Algorithms , 2014, Proceedings of the IEEE.

[56]  Pierre Hansen,et al.  NP-hardness of Euclidean sum-of-squares clustering , 2008, Machine Learning.

[57]  Adrien Gruson,et al.  Temporal coherency for video tone mapping , 2012, Other Conferences.