Expanding low dynamic range videos for high dynamic range applications

In this paper we introduce an algorithm and related methods that expand the contrast range of Low Dynamic Range (LDR) videos in order to regenerate missing High Dynamic Range (HDR) data. For content generated from single exposure LDR sequences, this is clearly an under constrained problem. We achieved the expansion by inverting established tone mapping operator, a process we term inverse tone mapping. This approach is augmented by a number of methods which help expand the luminance for the required pixels while avoiding artifacts. These methods may be used to convert the large libraries of available legacy LDR content for use, for instance, on new content-starved HDR devices. Moreover, these same methods may be used to provide animated emissive surfaces for image based lighting (IBL). We demonstrate results for all the above applications and validate the resultant HDR videos with original HDR references using the HDR Visual Difference Predictor (HDR-VDP) image metric.

[1]  Serge J. Belongie,et al.  Structured importance sampling of environment maps , 2003, ACM Trans. Graph..

[2]  Edward H. Adelson,et al.  Compressing and companding high dynamic range images with subband architectures , 2005, ACM Trans. Graph..

[3]  Scott J. Daly,et al.  Visible differences predictor: an algorithm for the assessment of image fidelity , 1992, Electronic Imaging.

[4]  Wolfgang Heidrich,et al.  High dynamic range display systems , 2004, SIGGRAPH 2004.

[5]  Wolfgang Heidrich,et al.  Ldr2Hdr: on-the-fly reverse tone mapping of legacy video and photographs , 2007, SIGGRAPH 2007.

[6]  Hayden Landis,et al.  Production-Ready Global Illumination , 2004 .

[7]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics) , 2005 .

[8]  Alan Hanjalic,et al.  Shot-boundary detection: unraveled and resolved? , 2002, IEEE Trans. Circuits Syst. Video Technol..

[9]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[10]  H. Jensen Realistic Image Synthesis Using Photon Mapping , 2001 .

[11]  Edward H. Adelson,et al.  Compressing and companding high dynamic range images with subband architectures , 2005, SIGGRAPH 2005.

[12]  Erik Reinhard,et al.  Do HDR displays support LDR content?: a psychophysical evaluation , 2007, ACM Trans. Graph..

[13]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.

[14]  Michael F. Cohen,et al.  Digital photography with flash and no-flash image pairs , 2004, ACM Trans. Graph..

[15]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[16]  Masahiro Okuda,et al.  Raw Image Encoding Based on Polynomial Approximation , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[17]  Laurence Meylan,et al.  The Reproduction of Specular Highlights on High Dynamic Range Displays , 2006, CIC.

[18]  David G. Stork,et al.  Pattern classification, 2nd Edition , 2000 .

[19]  Heung-Yeung Shum,et al.  Radiometric calibration from a single image , 2004, CVPR 2004.

[20]  Hans-Peter Seidel,et al.  Visible difference predicator for high dynamic range images , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[21]  J. H. van Hateren,et al.  Encoding of high dynamic range video with a model of human cones , 2006, TOGS.

[22]  Victor Ostromoukhov,et al.  Fast hierarchical importance sampling with blue noise properties , 2004, ACM Trans. Graph..

[23]  Hans-Peter Seidel,et al.  Backward compatible high dynamic range MPEG video compression , 2006, SIGGRAPH 2006.

[24]  Alexander Keller,et al.  Efficient Illumination by High Dynamic Range Images , 2003, Rendering Techniques.

[25]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[26]  Franklin C. Crow,et al.  Summed-area tables for texture mapping , 1984, SIGGRAPH.

[27]  Paul Debevec Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.

[28]  Shree K. Nayar,et al.  Radiometric self calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[29]  E. Reinhard Photographic Tone Reproduction for Digital Images , 2002 .

[30]  Frédo Durand,et al.  Flash photography enhancement via intrinsic relighting , 2004, SIGGRAPH 2004.

[31]  Paul E. Debevec,et al.  A median cut algorithm for light probe sampling , 2005, SIGGRAPH Courses.

[32]  Kun Zhou,et al.  High dynamic range image hallucination , 2007, SIGGRAPH '07.

[33]  Francesco Banterle,et al.  Inverse tone mapping , 2006, GRAPHITE '06.