Motion-free exposure fusion based on inter-consistency and intra-consistency

Exposure fusion often suffers from ghost artifacts, which are caused by the movement of objects when a dynamic scene is captured. In this paper, two types of consistency concepts are introduced for enforcing the guidance of a reference image for motion detection and ghost removal. Specifically, the inter-consistency, which represents the similarities of pixel intensities among different exposures, is weakened by the use of different exposure settings. Histogram matching is employed to recover the inter-consistency. Following this, pixel differences are mostly the result of changes in content caused by object movements, so motion can easily be detected. To further restrain the weights of outliers in fusion, motion detection is performed at a super-pixel level, to ensure that pixels with similar intensities and structures share similar fusion weights. This is referred to as intra-consistency. Experiments in various dynamic scenes demonstrate that the proposed algorithm can determine the motion more effectively than existing methods, and produce high quality fusion results that are free of ghost artifacts.

[2]  Jing Liu,et al.  Clustering-Guided Sparse Structural Learning for Unsupervised Feature Selection , 2014, IEEE Transactions on Knowledge and Data Engineering.

[3]  Greg Ward,et al.  Fast, Robust Image Registration for Compositing High Dynamic Range Photographs from Hand-Held Exposures , 2003, J. Graphics, GPU, & Game Tools.

[4]  E. Reinhard Photographic Tone Reproduction for Digital Images , 2002 .

[5]  Rae-Hong Park,et al.  Histogram based ghost removal in high dynamic range images , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[6]  Xiaosong Li,et al.  Multifocus image fusion by combining with mixed-order structure tensors and multiscale neighborhood , 2016, Inf. Sci..

[7]  Erik Reinhard,et al.  Ghost Removal in High Dynamic Range Images , 2006, 2006 International Conference on Image Processing.

[8]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[9]  Edward H. Adelson,et al.  Compressing and companding high dynamic range images with subband architectures , 2005, SIGGRAPH 2005.

[10]  William Puech,et al.  Ghost detection and removal in High Dynamic Range Images , 2009, 2009 17th European Signal Processing Conference.

[11]  Shiqian Wu,et al.  Selectively Detail-Enhanced Fusion of Differently Exposed Images With Moving Objects , 2014, IEEE Transactions on Image Processing.

[12]  Jing Liu,et al.  Robust Structured Subspace Learning for Data Representation , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Wai-kuen Cham,et al.  Gradient-Directed Multiexposure Composition , 2012, IEEE Transactions on Image Processing.

[14]  Meng Wang,et al.  Neighborhood Discriminant Hashing for Large-Scale Image Retrieval , 2015, IEEE Transactions on Image Processing.

[15]  Wai-kuen Cham,et al.  Gradient-directed composition of multi-exposure images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Edward H. Adelson,et al.  Compressing and companding high dynamic range images with subband architectures , 2005, ACM Trans. Graph..

[17]  Eli Shechtman,et al.  Robust patch-based hdr reconstruction of dynamic scenes , 2012, ACM Trans. Graph..

[18]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (The Morgan Kaufmann Series in Computer Graphics) , 2005 .

[19]  Dani Lischinski,et al.  Non-rigid dense correspondence with applications for image enhancement , 2011, ACM Trans. Graph..

[20]  Jitendra Malik,et al.  Recovering human body configurations: combining segmentation and recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[21]  Marius Tico,et al.  Artifact-free High Dynamic Range imaging , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[22]  Dani Lischinski,et al.  Gradient Domain High Dynamic Range Compression , 2023 .

[23]  Alexei A. Efros,et al.  Fast bilateral filtering for the display of high-dynamic-range images , 2002 .

[24]  Subhasis Chaudhuri,et al.  Bilateral Filter Based Compositing for Variable Exposure Photography , 2009, Eurographics.

[25]  Adam Finkelstein,et al.  The Generalized PatchMatch Correspondence Algorithm , 2010, ECCV.

[26]  Joachim Weickert,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Freehand Hdr Imaging of Moving Scenes with Simultaneous Resolution Enhancement Freehand Hdr Imaging of Moving Scenes with Simultaneous Resolution Enhancement Freehand Hdr Imaging of Moving Scenes with Simultaneous Resolution Enhancement , 2022 .

[27]  Jan Kautz,et al.  Bitmap Movement Detection: HDR for Dynamic Scenes , 2010 .

[28]  Jun Hu,et al.  Exposure Stacks of Live Scenes with Hand-Held Cameras , 2012, ECCV.

[29]  A·H·埃尔-玛迪,et al.  High-dynamic range video tone mapping , 2011 .

[30]  Sang Uk Lee,et al.  Ghost-Free High Dynamic Range Imaging , 2010, ACCV.

[31]  Thorsten Grosch,et al.  Fast and Robust High Dynamic Range Image Generation with Camera and Object Movement , 2006 .

[32]  Shree K. Nayar,et al.  Determining the Camera Response from Images: What Is Knowable? , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Edward Y. Chang,et al.  A data-driven study of image feature extraction and fusion , 2014, Inf. Sci..

[34]  Jun Hu,et al.  HDR Deghosting: How to Deal with Saturation? , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Greg Ward,et al.  Automatic High-Dynamic Range Image Generation for Dynamic Scenes , 2008, IEEE Computer Graphics and Applications.

[36]  Desire Sidibé,et al.  Ghost detection and removal for high dynamic range images: Recent advances , 2012, Signal Process. Image Commun..

[37]  Janne Heikkilä,et al.  Constrain Propagation for Ghost Removal in High Dynamic Range Images , 2008, VISAPP.

[38]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[39]  Jan Kautz,et al.  Exposure Fusion , 2009, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[40]  Susanto Rahardja,et al.  Movement detection for the synthesis of high dynamic range images , 2010, 2010 IEEE International Conference on Image Processing.

[41]  Wai-kuen Cham,et al.  Reference-guided exposure fusion in dynamic scenes , 2012, J. Vis. Commun. Image Represent..