Multi-Exposure Fusion with CNN Features

Multi-exposure fusion (MEF) is a widely used approach to high dynamic range imaging. The selection of features for fusion weight calculation is important to the performance of MEF. In this paper, we investigate the effectiveness of convolutional neural network (CNN) features for MEF. Considering the fact that there are no ground-truth images in MEF to train an end-to-end CNN, we adopt the pre-trained networks in other tasks to extract the feature. Both the selection of network and the selection of convolution layer are studied. With the extracted CNN feature map, we compute the local visibility and consistency maps to determine the weight map for MEF. The proposed method works well for both static and dynamic scenes. It exhibits competitive quantitative measures, and presents perceptually pleasing MEF outputs with little halo effects.

[1]  Shutao Li,et al.  Image Fusion With Guided Filtering , 2013, IEEE Transactions on Image Processing.

[2]  Subhasis Chaudhuri,et al.  Bilateral Filter Based Compositing for Variable Exposure Photography , 2009, Eurographics.

[3]  Lei Zhang,et al.  Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images , 2018, IEEE Transactions on Image Processing.

[4]  Kyoung Mu Lee,et al.  Accurate Image Super-Resolution Using Very Deep Convolutional Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Kai Zeng,et al.  Perceptual Quality Assessment for Multi-Exposure Image Fusion , 2015, IEEE Transactions on Image Processing.

[6]  Xuelong Li,et al.  Exposure Fusion Using Boosting Laplacian Pyramid , 2014, IEEE Transactions on Cybernetics.

[7]  Weidong Min,et al.  Remote Sensing Image Registration Using Convolutional Neural Network Features , 2018, IEEE Geoscience and Remote Sensing Letters.

[8]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.

[9]  Jan Kautz,et al.  Exposure Fusion , 2009, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[10]  Zhengfang Duanmu,et al.  Multi-Exposure Image Fusion by Optimizing A Structural Similarity Index , 2018, IEEE Transactions on Computational Imaging.

[11]  Eli Shechtman,et al.  Robust patch-based hdr reconstruction of dynamic scenes , 2012, ACM Trans. Graph..

[12]  Susanto Rahardja,et al.  Detail-Enhanced Exposure Fusion , 2012, IEEE Transactions on Image Processing.

[13]  Shutao Li,et al.  Fast multi-exposure image fusion with median filter and recursive filter , 2012, IEEE Transactions on Consumer Electronics.

[14]  Lei Zhang,et al.  Clustering based content and color adaptive tone mapping , 2017, Comput. Vis. Image Underst..

[15]  R. Venkatesh Babu,et al.  DeepFuse: A Deep Unsupervised Approach for Exposure Fusion with Extreme Exposure Image Pairs , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[16]  Shiqian Wu,et al.  Selectively Detail-Enhanced Fusion of Differently Exposed Images With Moving Objects , 2014, IEEE Transactions on Image Processing.

[17]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[18]  Marius Tico,et al.  Artifact-free High Dynamic Range imaging , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[19]  Yu Liu,et al.  Dense SIFT for ghost-free multi-exposure fusion , 2015, J. Vis. Commun. Image Represent..

[20]  Zhengguo Li,et al.  Multi-scale exposure fusion via gradient domain guided image filtering , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[21]  A. Ardeshir Goshtasby,et al.  Fusion of multi-exposure images , 2005, Image Vis. Comput..

[22]  Bo Gu,et al.  Gradient field multi-exposure images fusion for high dynamic range image visualization , 2012, J. Vis. Commun. Image Represent..

[23]  Kai Zeng,et al.  High Dynamic Range Image Compression by Optimizing Tone Mapped Image Quality Index , 2015, IEEE Transactions on Image Processing.

[24]  Zhou Wang,et al.  Multi-exposure image fusion: A patch-wise approach , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[25]  Luc Van Gool,et al.  DSLR-Quality Photos on Mobile Devices with Deep Convolutional Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Lei Zhang,et al.  Robust Multi-Exposure Image Fusion: A Structural Patch Decomposition Approach , 2017, IEEE Transactions on Image Processing.

[27]  Wai-kuen Cham,et al.  Gradient-Directed Multiexposure Composition , 2012, IEEE Transactions on Image Processing.