Noise-Aware Merging of High Dynamic Range Image Stacks without Camera Calibration

A near-optimal reconstruction of the radiance of a High Dynamic Range scene from an exposure stack can be obtained by modeling the camera noise distribution. The latent radiance is then estimated using Maximum Likelihood Estimation. But this requires a well-calibrated noise model of the camera, which is difficult to obtain in practice. We show that an unbiased estimation of comparable variance can be obtained with a simpler Poisson noise estimator, which does not require the knowledge of camera-specific noise parameters. We demonstrate this empirically for four different cameras, ranging from a smartphone camera to a full-frame mirrorless camera. Our experimental results are consistent for simulated as well as real images, and across different camera settings.

[1]  J. Delon,et al.  Study of the digital camera acquisition process and statistical modeling of the sensor raw data , 2013 .

[2]  Shree K. Nayar,et al.  Radiometric self calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Anna Tomaszewska,et al.  Image Registration for Multi-exposure High Dynamic Range Image Acquisition , 2007 .

[4]  Glenn Healey,et al.  Radiometric CCD camera calibration and noise estimation , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Hans-Peter Seidel,et al.  Optimal HDR reconstruction with linear digital cameras , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[6]  Jonathan T. Barron,et al.  Jump: virtual reality video , 2016, ACM Trans. Graph..

[7]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[8]  Mikhail V. Konnik,et al.  High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial , 2014, ArXiv.

[9]  Takeo Kanade,et al.  Statistical Calibration of the CCD Imaging Process , 2001, ICCV.

[10]  Aykut Erdem,et al.  The State of the Art in HDR Deghosting: A Survey and Evaluation , 2015, Comput. Graph. Forum.

[11]  Jonas Unger,et al.  Adaptive dualISO HDR reconstruction , 2015, EURASIP J. Image Video Process..

[12]  Roberto Costantini,et al.  Virtual sensor design , 2004, IS&T/SPIE Electronic Imaging.

[13]  Hans Jørgen Andersen,et al.  Noise Characterization of Weighting Schemes for Combination of Multiple Exposures , 2006, BMVC.

[14]  Julie Delon,et al.  Best Algorithms for HDR Image Generation. A Study of Performance Bounds , 2014, SIAM J. Imaging Sci..

[15]  Robert L. Stevenson,et al.  Estimation-theoretic approach to dynamic range enhancement using multiple exposures , 2003, J. Electronic Imaging.

[16]  Joachim Weickert,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Optic Flow in Harmony Optic Flow in Harmony Optic Flow in Harmony , 2022 .

[17]  Erik Reinhard,et al.  Noise reduction in high dynamic range imaging , 2007, J. Vis. Commun. Image Represent..

[18]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, ACM Trans. Graph..

[19]  Suren Jayasuriya,et al.  Reconfiguring the Imaging Pipeline for Computer Vision , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Jeffrey C. Lagarias,et al.  Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions , 1998, SIAM J. Optim..

[21]  Steve Mann,et al.  ON BEING `UNDIGITAL' WITH DIGITAL CAMERAS: EXTENDING DYNAMIC RANGE BY COMBINING DIFFERENTLY EXPOSED PICTURES , 1995 .

[22]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[23]  Orazio Gallo,et al.  Stack-Based Algorithms for HDR Capture and Reconstruction , 2016 .

[24]  Frédo Durand,et al.  Noise-optimal capture for high dynamic range photography , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[25]  Elsevier Sdol,et al.  Journal of Visual Communication and Image Representation , 2009 .

[26]  Kanita Karaduzovi Hadziabdic,et al.  Expert evaluation of deghosting algorithms for multi-exposure high dynamic range imaging , 2014 .