Robust patch-based hdr reconstruction of dynamic scenes

High dynamic range (HDR) imaging from a set of sequential exposures is an easy way to capture high-quality images of static scenes, but suffers from artifacts for scenes with significant motion. In this paper, we propose a new approach to HDR reconstruction that draws information from all the exposures but is more robust to camera/scene motion than previous techniques. Our algorithm is based on a novel patch-based energy-minimization formulation that integrates alignment and reconstruction in a joint optimization through an equation we call the HDR image synthesis equation. This allows us to produce an HDR result that is aligned to one of the exposures yet contains information from all of them. We present results that show considerable improvement over previous approaches.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[3]  Steve Mann,et al.  ON BEING `UNDIGITAL' WITH DIGITAL CAMERAS: EXTENDING DYNAMIC RANGE BY COMBINING DIFFERENTLY EXPOSED PICTURES , 1995 .

[4]  Shree K. Nayar,et al.  Radiometric self calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[5]  Luca Bogoni,et al.  Extending dynamic range of monochrome and color images through fusion , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[6]  Shree K. Nayar,et al.  High dynamic range imaging: spatially varying pixel exposures , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[7]  Greg Ward,et al.  Fast, Robust Image Registration for Compositing High Dynamic Range Photographs from Hand-Held Exposures , 2003, J. Graphics, GPU, & Game Tools.

[8]  Abbas El Gamal,et al.  Synthesis of high dynamic range motion blur free image from multiple captures , 2003 .

[9]  Richard Szeliski,et al.  High dynamic range video , 2003, ACM Trans. Graph..

[10]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[11]  Stephen Lin,et al.  Radiometric calibration from a single image , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[12]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[13]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[14]  Joachim Weickert,et al.  Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods , 2005, International Journal of Computer Vision.

[15]  Stephen Lin,et al.  Determining the radiometric response function from a single grayscale image , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  J. Weickert,et al.  Lucas/Kanade meets Horn/Schunck: combining local and global optic flow methods , 2005 .

[17]  Erik Reinhard,et al.  Ghost Removal in High Dynamic Range Images , 2006, 2006 International Conference on Image Processing.

[18]  Thorsten Grosch,et al.  Fast and Robust High Dynamic Range Image Generation with Camera and Object Movement , 2006 .

[19]  Richard Szeliski,et al.  Seamless Image Stitching of Scenes with Large Motions and Exposure Differences , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[20]  Eli Shechtman,et al.  Space-Time Completion of Video , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Richard Szeliski,et al.  A Database and Evaluation Methodology for Optical Flow , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[22]  Anna Tomaszewska,et al.  Image Registration for Multi-exposure High Dynamic Range Image Acquisition , 2007 .

[23]  Masahiro Okuda,et al.  Motion blur free HDR image acquisition using multiple exposures , 2008, 2008 15th IEEE International Conference on Image Processing.

[24]  Denis Simakov,et al.  Summarizing visual data using bidirectional similarity , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[26]  Greg Ward,et al.  Automatic High-Dynamic Range Image Generation for Dynamic Scenes , 2008, IEEE Computer Graphics and Applications.

[27]  Adam Finkelstein,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, SIGGRAPH 2009.

[28]  William Puech,et al.  Ghost detection and removal in High Dynamic Range Images , 2009, 2009 17th European Signal Processing Conference.

[29]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[30]  Huei-Yung Lin,et al.  High dynamic range imaging for stereoscopic scene representation , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[31]  Rae-Hong Park,et al.  Histogram based ghost removal in high dynamic range images , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[32]  Marius Tico,et al.  Artifact-free High Dynamic Range imaging , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[33]  Jan Kautz,et al.  Bitmap Movement Detection: HDR for Dynamic Scenes , 2010, 2010 Conference on Visual Media Production.

[34]  Eli Shechtman,et al.  Regenerative morphing , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[36]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[37]  Sang Uk Lee,et al.  Ghost-Free High Dynamic Range Imaging , 2010, ACCV.

[38]  Stephen Mangiat,et al.  High dynamic range video with ghost removal , 2010, Optical Engineering + Applications.

[39]  Susanto Rahardja,et al.  A robust and fast anti-ghosting algorithm for high dynamic range imaging , 2010, 2010 IEEE International Conference on Image Processing.

[40]  J Sullivan Gary,et al.  高効率ビデオコーディング(HEVC)標準化における最近の進展 , 2010 .

[41]  Subhasis Chaudhuri,et al.  Reconstruction of high contrast images for dynamic scenes , 2011, The Visual Computer.

[42]  Kurt Debattista,et al.  Advanced High Dynamic Range Imaging: Theory and Practice , 2011 .

[43]  Ahmet Oguz Akyüzy.  Photographically Guided Alignment for HDR Images , 2011 .

[44]  Ahmet Oguz Akyüz.  Photographically Guided Alignment for HDR Images , 2011, Eurographics.

[45]  Susu Yao.  Robust image registration for multiple exposure high dynamic range image synthesis , 2011, Electronic Imaging.

[46]  Joachim Weickert,et al.  Freehand HDR Imaging of Moving Scenes with Simultaneous Resolution Enhancement , 2011, Comput. Graph. Forum.

[47]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[48]  Dani Lischinski,et al.  Non-rigid dense correspondence with applications for image enhancement , 2011, ACM Trans. Graph..

[49]  Jitendra Malik,et al.  Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[50]  Yasuyuki Matsushita,et al.  Motion detail preserving optical flow estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  Wai-kuen Cham,et al.  Gradient-Directed Multiexposure Composition , 2012, IEEE Transactions on Image Processing.