Overview of State-of-the-Art Algorithms for Stack-Based High-Dynamic Range (HDR) Imaging

Modern digital cameras have very limited dynamic range, which makes them unable to capture the full range of illumination in natural scenes. Since this prevents them from accurately photographing visible detail, researchers have spent the last two decades developing algorithms for high-dynamic range (HDR) imaging which can capture a wider range of illumination and therefore allow us to reconstruct richer images of natural scenes. The most practical of these methods are stack-based approaches which take a set of images at different exposure levels and then merge them together to form the final HDR result. However, these algorithms produce ghost-like artifacts when the scene has motion or the camera is not perfectly static. In this paper, we present an overview of state-of-the-art deghosting algorithms for stackbased HDR imaging and discuss some of the tradeoffs of each.

[1]  Greg Ward,et al.  Fast, Robust Image Registration for Compositing High Dynamic Range Photographs from Hand-Held Exposures , 2003, J. Graphics, GPU, & Game Tools.

[2]  Anna Tomaszewska,et al.  Image Registration for Multi-exposure High Dynamic Range Image Acquisition , 2007 .

[3]  B. C. Madden,et al.  Extended Intensity Range Imaging , 1993 .

[4]  Erik Reinhard,et al.  Ghost Removal in High Dynamic Range Images , 2006, 2006 International Conference on Image Processing.

[5]  Steve Mann,et al.  ON BEING `UNDIGITAL' WITH DIGITAL CAMERAS: EXTENDING DYNAMIC RANGE BY COMBINING DIFFERENTLY EXPOSED PICTURES , 1995 .

[6]  Jun Hu,et al.  Exposure Stacks of Live Scenes with Hand-Held Cameras , 2012, ECCV.

[7]  Sang Uk Lee,et al.  Ghost-Free High Dynamic Range Imaging , 2010, ACCV.

[8]  Aykut Erdem,et al.  The State of the Art in HDR Deghosting: A Survey and Evaluation , 2015, Comput. Graph. Forum.

[9]  Jan Kautz,et al.  Bitmap Movement Detection: HDR for Dynamic Scenes , 2010, 2010 Conference on Visual Media Production.

[10]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[11]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[12]  Jun Hu,et al.  HDR Deghosting: How to Deal with Saturation? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[14]  Susanto Rahardja,et al.  A robust and fast anti-ghosting algorithm for high dynamic range imaging , 2010, 2010 IEEE International Conference on Image Processing.

[15]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics) , 2005 .

[16]  Dani Lischinski,et al.  Non-rigid dense correspondence with applications for image enhancement , 2011, ACM Trans. Graph..

[17]  Stephen Mangiat,et al.  High dynamic range video with ghost removal , 2010, Optical Engineering + Applications.

[18]  Orazio Gallo,et al.  Stack-Based Algorithms for HDR Capture and Reconstruction , 2016 .

[19]  Eli Shechtman,et al.  Image melding , 2012, ACM Trans. Graph..

[20]  Thorsten Grosch,et al.  Fast and Robust High Dynamic Range Image Generation with Camera and Object Movement , 2006 .

[21]  Ravi Ramamoorthi,et al.  Deep high dynamic range imaging of dynamic scenes , 2017, ACM Trans. Graph..

[22]  Li Zhang,et al.  Denoising vs. deblurring: HDR imaging techniques using moving cameras , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Shree K. Nayar,et al.  Determining the Camera Response from Images: What Is Knowable? , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Rae-Hong Park,et al.  Histogram based ghost removal in high dynamic range images , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[25]  Joachim Weickert,et al.  Freehand HDR Imaging of Moving Scenes with Simultaneous Resolution Enhancement , 2011, Comput. Graph. Forum.

[26]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[27]  Pradeep Sen,et al.  Robust Radiometric Calibration for Dynamic Scenes in the Wild , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[28]  William Puech,et al.  Ghost detection and removal in High Dynamic Range Images , 2009, 2009 17th European Signal Processing Conference.

[29]  Greg Ward,et al.  Automatic High-Dynamic Range Image Generation for Dynamic Scenes , 2008, IEEE Computer Graphics and Applications.

[30]  Desire Sidibé,et al.  Ghost detection and removal for high dynamic range images: Recent advances , 2012, Signal Process. Image Commun..

[31]  Eli Shechtman,et al.  Patch-based high dynamic range video , 2013, ACM Trans. Graph..

[32]  Tae-Hyun Oh,et al.  Robust High Dynamic Range Imaging by Rank Minimization , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Richard Szeliski,et al.  High dynamic range video , 2003, ACM Trans. Graph..

[34]  Masahiro Okuda,et al.  Motion blur free HDR image acquisition using multiple exposures , 2008, 2008 15th IEEE International Conference on Image Processing.

[35]  Wai-kuen Cham,et al.  Reference-guided exposure fusion in dynamic scenes , 2012, J. Vis. Commun. Image Represent..

[36]  Luca Bogoni,et al.  Extending dynamic range of monochrome and color images through fusion , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[37]  Jerry D. Gibson,et al.  Spatially adaptive filtering for registration artifact removal in HDR video , 2011, 2011 18th IEEE International Conference on Image Processing.

[38]  Jan Kautz,et al.  Exposure Fusion , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[39]  Jonathan T. Barron,et al.  Burst photography for high dynamic range and low-light imaging on mobile cameras , 2016, ACM Trans. Graph..

[40]  Richard Szeliski,et al.  Seamless Image Stitching of Scenes with Large Motions and Exposure Differences , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[41]  Shree K. Nayar,et al.  High dynamic range imaging: spatially varying pixel exposures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[42]  Miguel Granados,et al.  Automatic noise modeling for ghost-free HDR reconstruction , 2013, ACM Trans. Graph..

[43]  Pradeep Sen,et al.  Practical High Dynamic Range Imaging of Everyday Scenes: Photographing the world as we see it with our own eyes , 2016, IEEE Signal Processing Magazine.

[44]  Gabriel Eilertsen,et al.  HDR image reconstruction from a single exposure using deep CNNs , 2017, ACM Trans. Graph..

[45]  Eli Shechtman,et al.  Robust patch-based hdr reconstruction of dynamic scenes , 2012, ACM Trans. Graph..

[46]  Denis Simakov,et al.  Summarizing visual data using bidirectional similarity , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Subhasis Chaudhuri,et al.  Reconstruction of high contrast images for dynamic scenes , 2011, The Visual Computer.

[48]  Jun Hu,et al.  Locally non-rigid registration for mobile HDR photography , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[49]  Rafal Mantiuk,et al.  Assessment of multi-exposure HDR image deghosting methods , 2017, Comput. Graph..