Spatio-temporal constrained tone mapping operator for HDR video compression

Abstract With the growing popularity of high dynamic range (HDR) imaging, efficient compression techniques are demanded, as HDR video entails typically higher raw data rate than traditional video. For this purpose, we introduce a hybrid spatially and temporally constrained content-adaptive tone mapping operator (TMO) to convert the input HDR video into a tone mapped video sequence, which is then encoded using the high efficiency video coding (HEVC) standard. The proposed TMO simultaneously exploits intra-frame spatial redundancies and preserves inter-frame temporal coherence of the tone mapped video sequence. Extensive experimental results show that the developed spatio-temporal TMO (ST-TMO) solution yields higher coding performance than existing frame-by-frame TMO’s, and compares favorably with state-of-the-art methods based on a fixed transfer function.

[1]  Yan Ye,et al.  Recent developments from MPEG in HDR video compression , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[2]  Herbert Thoma,et al.  Temporally coherent luminance-to-luma mapping for high dynamic range video coding with H.264/AVC , 2011, 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[3]  S. Osher,et al.  Image restoration: Total variation, wavelet frames, and beyond , 2012 .

[4]  Adrien Gruson,et al.  Temporal coherency for video tone mapping , 2012, Other Conferences.

[5]  Rabab Kreidieh Ward,et al.  Optimizing a Tone Curve for Backward-Compatible High Dynamic Range Image and Video Compression , 2011, IEEE Transactions on Image Processing.

[6]  Dominique Thoreau,et al.  Zonal brightness coherency for video tone mapping , 2014, Signal Process. Image Commun..

[7]  Christine Guillemot,et al.  Adaptive re-quantization for high dynamic range video compression , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  Tim Borer Non-linear Opto-Electrical Transfer Functions for High Dynamic Range Television , 2014 .

[9]  Kurt Debattista,et al.  HDR video past, present and future: A perspective , 2017, Signal Process. Image Commun..

[10]  Giuseppe Valenzise,et al.  HDR Video Coding based on a temporally constrained Tone Mapping Operator , 2016, 2016 Digital Media Industry & Academic Forum (DMIAF).

[11]  Giuseppe Valenzise,et al.  Dynamic range expansion of video sequences: A subjective quality assessment study , 2014, 2014 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[12]  Herbert Thoma,et al.  An adaptive Logluv transform for High Dynamic Range video compression , 2010, 2010 IEEE International Conference on Image Processing.

[13]  Rabab K. Ward,et al.  Visually Favorable Tone-Mapping With High Compression Performance in Bit-Depth Scalable Video Coding , 2013, IEEE Trans. Multim..

[14]  Patrick Le Callet,et al.  High Dynamic Range Video - From Acquisition, to Display and Applications , 2016 .

[15]  Touradj Ebrahimi,et al.  Benchmarking of objective quality metrics for HDR image quality assessment , 2015, EURASIP Journal on Image and Video Processing.

[16]  S. Mallat A wavelet tour of signal processing , 1998 .

[17]  Heinz H. Bauschke,et al.  Fixed-Point Algorithms for Inverse Problems in Science and Engineering , 2011, Springer Optimization and Its Applications.

[18]  Christine D. Piatko,et al.  A visibility matching tone reproduction operator for high dynamic range scenes , 1997 .

[19]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[20]  Patrick Le Callet,et al.  HDR-VDP-2.2: a calibrated method for objective quality prediction of high-dynamic range and standard images , 2014, J. Electronic Imaging.

[21]  E. Reinhard Photographic Tone Reproduction for Digital Images , 2002 .

[22]  K. R. Rao,et al.  High Efficiency Video Coding(HEVC) , 2014 .

[23]  W. Marsden I and J , 2012 .

[24]  Erik Reinhard,et al.  Perception-based high dynamic range video compression with optimal bit-depth transformation , 2011, 2011 18th IEEE International Conference on Image Processing.

[25]  Panos Nasiopoulos,et al.  Demystifying High-Dynamic-Range Technology: A new evolution in digital media. , 2015, IEEE Consumer Electronics Magazine.

[26]  Rafal Mantiuk,et al.  Display adaptive tone mapping , 2008, SIGGRAPH 2008.

[27]  Aljoscha Smolic,et al.  Suplemental Material for Temporally Coherent Local Tone Mapping of HDR Video , 2014 .

[28]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[29]  Karol Myszkowski,et al.  Adaptive Logarithmic Mapping For Displaying High Contrast Scenes , 2003, Comput. Graph. Forum.

[30]  Emin Zerman,et al.  An extensive performance evaluation of full-reference HDR image quality metrics , 2017 .

[31]  Giuseppe Valenzise,et al.  Effect of color space on high dynamic range video compression performance , 2017, 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX).

[32]  Antonin Chambolle,et al.  A First-Order Primal-Dual Algorithm for Convex Problems with Applications to Imaging , 2011, Journal of Mathematical Imaging and Vision.

[33]  J. Pesquet,et al.  A Parallel Inertial Proximal Optimization Method , 2012 .

[34]  Ajay Luthra,et al.  High Dynamic Range and Wide Color Gamut Video Coding in HEVC: Status and Potential Future Enhancements , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Scott Miller,et al.  Perceptual Signal Coding for More Efficient Usage of Bit Codes , 2012 .

[36]  Peter G. J. Barten,et al.  Formula for the contrast sensitivity of the human eye , 2003, IS&T/SPIE Electronic Imaging.

[37]  G. Bjontegaard,et al.  Calculation of Average PSNR Differences between RD-curves , 2001 .

[38]  David R. Bull,et al.  High dynamic range video compression by intensity dependent spatial quantization in HEVC , 2013, 2013 Picture Coding Symposium (PCS).

[39]  Patrick Le Callet,et al.  HDR-VQM: An objective quality measure for high dynamic range video , 2015, Signal Process. Image Commun..

[40]  Giuseppe Valenzise,et al.  Improved tone mapping operator for HDR coding optimizing the distortion/spatial complexity trade-off , 2014, 2014 22nd European Signal Processing Conference (EUSIPCO).

[41]  Hans-Peter Seidel,et al.  A perceptual framework for contrast processing of high dynamic range images , 2006, TAP.

[42]  Nelly Pustelnik,et al.  Epigraphical splitting for solving constrained convex formulations of inverse problems with proximal tools , 2012, 1210.5844.

[43]  Frédéric Dufaux,et al.  Methods for improving the tone mapping for backward compatible high dynamic range image and video coding , 2014, Signal Process. Image Commun..

[44]  David R. Bull,et al.  High Dynamic Range Video Compression Exploiting Luminance Masking , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[45]  Giovanni Chierchia,et al.  Parallel implementations of a disparity estimation algorithm based on a Proximal splitting method , 2012, 2012 Visual Communications and Image Processing.

[46]  P. L. Combettes,et al.  Primal-Dual Splitting Algorithm for Solving Inclusions with Mixtures of Composite, Lipschitzian, and Parallel-Sum Type Monotone Operators , 2011, Set-Valued and Variational Analysis.

[47]  Karol Myszkowski,et al.  High Dynamic Range Video , 2008, Synthesis Lectures on Computer Graphics and Animation.

[48]  Dominique Thoreau,et al.  Impact of temporal coherence-based Tone Mapping on video compression , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[49]  Gregory Ward Larson,et al.  Overcoming Gamut and Dynamic Range Limitations in Digital Images , 1998, CIC.