Dual-Attention-Guided Network for Ghost-Free High Dynamic Range Imaging

Ghosting artifacts caused by moving objects and misalignments are a key challenge in constructing high dynamic range (HDR) images. Current methods first register the input low dynamic range (LDR) images using optical flow before merging them. This process is error-prone, and often causes ghosting in the resulting merged image. We propose a novel dual-attention-guided end-to-end deep neural network, called DAHDRNet, which produces high-quality ghost-free HDR images. Unlike previous methods that directly stack the LDR images or features for merging, we use dual-attention modules to guide the merging according to the reference image. DAHDRNet thus exploits both spatial attention and feature channel attention to achieve ghost-free merging. The spatial attention modules automatically suppress undesired components caused by misalignments and saturation, and enhance the fine details in the non-reference images. The channel attention modules adaptively rescale channel-wise features by considering the inter-dependencies between channels. The dual-attention approach is applied recurrently to further improve feature representation, and thus alignment. A dilated residual dense block is devised to make full use of the hierarchical features and increase the receptive field when hallucinating missing details. We employ a hybrid loss function, which consists of a perceptual loss, a total variation loss, and a content loss to recover photo-realistic images. Although DAHDRNet is not flow-based, it can be applied to flow-based registration to reduce artifacts caused by optical-flow estimation errors. Experiments on different datasets show that the proposed DAHDRNet achieves state-of-the-art quantitative and qualitative results.

[1]  Subhasis Chaudhuri,et al.  Reconstruction of high contrast images for dynamic scenes , 2011, The Visual Computer.

[2]  Yun Fu,et al.  Image Super-Resolution Using Very Deep Residual Channel Attention Networks , 2018, ECCV.

[3]  Chi-Keung Tang,et al.  Deep High Dynamic Range Imaging with Large Foreground Motions , 2017, ECCV.

[4]  Anders P. Eriksson,et al.  Sampson distance based joint estimation of multiple homographies with uncalibrated cameras , 2014, Comput. Vis. Image Underst..

[5]  Sang Uk Lee,et al.  Ghost-Free High Dynamic Range Imaging , 2010, ACCV.

[6]  Chul Lee,et al.  Ghost-Free High Dynamic Range Imaging via Rank Minimization , 2014, IEEE Signal Processing Letters.

[7]  Ing Ren Tsang,et al.  Single image HDR reconstruction using a CNN with masked features and perceptual loss , 2020, ACM Trans. Graph..

[8]  Tae-Hyun Oh,et al.  Robust High Dynamic Range Imaging by Rank Minimization , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Gabriel Eilertsen,et al.  HDR image reconstruction from a single exposure using deep CNNs , 2017, ACM Trans. Graph..

[10]  Lei Zhang,et al.  Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images , 2018, IEEE Transactions on Image Processing.

[11]  Yu Zhu,et al.  Deep HDR Imaging via A Non-Local Network , 2020, IEEE Transactions on Image Processing.

[12]  R. Venkatesh Babu,et al.  Towards Practical and Efficient High-Resolution HDR Deghosting with CNN , 2020, ECCV.

[13]  Joachim Weickert,et al.  Freehand HDR Imaging of Moving Scenes with Simultaneous Resolution Enhancement , 2011, Comput. Graph. Forum.

[14]  Yoshihiro Kanamori,et al.  Deep reverse tone mapping , 2017, ACM Trans. Graph..

[15]  Eli Shechtman,et al.  Robust patch-based hdr reconstruction of dynamic scenes , 2012, ACM Trans. Graph..

[16]  Jonathan T. Barron,et al.  Deep bilateral learning for real-time image enhancement , 2017, ACM Trans. Graph..

[17]  Ravi Ramamoorthi,et al.  Deep HDR Video from Sequences with Alternating Exposures , 2019, Comput. Graph. Forum.

[18]  Ravi Ramamoorthi,et al.  Deep high dynamic range imaging of dynamic scenes , 2017, ACM Trans. Graph..

[19]  Yanning Zhang,et al.  High dynamic range imaging by sparse representation , 2017, Neurocomputing.

[20]  Aykut Erdem,et al.  An Objective Deghosting Quality Metric for HDR Images , 2016, Comput. Graph. Forum.

[21]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH.

[22]  Wolfgang Heidrich,et al.  HDR-VDP-2: a calibrated visual metric for visibility and quality predictions in all luminance conditions , 2011, SIGGRAPH 2011.

[23]  Greg Ward,et al.  Automatic High-Dynamic Range Image Generation for Dynamic Scenes , 2008, IEEE Computer Graphics and Applications.

[24]  Aykut Erdem,et al.  The State of the Art in HDR Deghosting: A Survey and Evaluation , 2015, Comput. Graph. Forum.

[25]  Gordon Wetzstein,et al.  Neural Sensors: Learning Pixel Exposures for HDR Imaging and Video Compressive Sensing With Programmable Sensors , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Jie Yang,et al.  Seeing Deeply and Bidirectionally: A Deep Learning Approach for Single Image Reflection Removal , 2018, ECCV.

[27]  Wai-kuen Cham,et al.  Gradient-Directed Multiexposure Composition , 2012, IEEE Transactions on Image Processing.