Multi-Exposure Decomposition-Fusion Model for High Dynamic Range Image Saliency Detection

High dynamic range (HDR) imaging techniques have witnessed a great improvement in the past few decades. However, saliency detection task on HDR content is still far from well explored. In this paper, we introduce a multi-exposure decomposition-fusion model for HDR image saliency detection inspired by the brightness adaption mechanism. The proposed model is composed of three modules. Firstly, a decomposition module converts the input raw HDR image into a stack of LDR images by uniformly sampling the exposure time range. Secondly, a saliency region proposal network is employed to generate the candidate saliency maps for each LDR image in the exposure stack. Finally, an uncertainty weighting based fusion algorithm is applied to generate the overall saliency map for the input HDR image by merging the obtained LDR saliency maps. Extensive experiments show that our proposed model achieves superior performance compared with the state-of-the-art methods on the existing HDR eye fixation databases. The source code of the proposed model are made publicly available at https://github.com/sunnycia/DFHSal.

[1]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[2]  Lei Zhang,et al.  Learning a Deep Single Image Contrast Enhancer from Multi-Exposure Images , 2018, IEEE Transactions on Image Processing.

[3]  Jan Kautz,et al.  Exposure Fusion , 2009, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[4]  Qi Zhao,et al.  SALICON: Reducing the Semantic Gap in Saliency Prediction by Adapting Deep Neural Networks , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[5]  Xavier Otazu,et al.  Which tone-mapping operator is the best? A comparative study of perceptual quality , 2016, Journal of the Optical Society of America. A, Optics, image science, and vision.

[6]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[7]  Erik Reinhard,et al.  Photographic tone reproduction for digital images , 2002, ACM Trans. Graph..

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Christine D. Piatko,et al.  A visibility matching tone reproduction operator for high dynamic range scenes , 1997, SIGGRAPH '97.

[10]  Ali Borji,et al.  CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research , 2015, ArXiv.

[11]  Aykut Erdem,et al.  Visual saliency estimation by nonlinearly integrating features using region covariances. , 2013, Journal of vision.

[12]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[13]  Manish Narwaria,et al.  Effect of tone mapping operators on visual attention deployment , 2012, Other Conferences.

[14]  R. Watt.  Scanning from coarse to fine spatial scales in the human visual system after the onset of a stimulus. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[15]  Min H. Kim,et al.  Modeling human color perception under extended luminance levels , 2009, ACM Trans. Graph..

[16]  Pietro Perona,et al.  Graph-Based Visual Saliency , 2006, NIPS.

[17]  Rita Cucchiara,et al.  Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model , 2016, IEEE Transactions on Image Processing.

[18]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Jan Kautz,et al.  Consistent tone reproduction , 2008 .

[20]  Rainer Goebel,et al.  Contextual Encoder-Decoder Network for Visual Saliency Prediction , 2019, Neural Networks.

[21]  Roberto Manduchi,et al.  Metering for Exposure Stacks , 2012, Comput. Graph. Forum.

[22]  Rabab Kreidieh Ward,et al.  Optimizing a Tone Curve for Backward-Compatible High Dynamic Range Image and Video Compression , 2011, IEEE Transactions on Image Processing.

[23]  Asha Iyer,et al.  Components of bottom-up gaze allocation in natural images , 2005, Vision Research.

[24]  Touradj Ebrahimi,et al.  Visual attention in LDR and HDR images , 2015 .

[25]  Panos Nasiopoulos,et al.  Human Visual System-Based Saliency Detection for High Dynamic Range Content , 2016, IEEE Transactions on Multimedia.

[26]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[27]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[28]  Qi Zhao,et al.  SALICON: Saliency in Context , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Hans-Peter Seidel,et al.  Perceptual effects in real-time tone mapping , 2005, SCCG '05.

[30]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[31]  Jean-Philippe Tarel,et al.  Saliency maps of high dynamic range images , 2009, APGV '09.

[32]  Qingming Huang,et al.  Image Saliency Detection Video Saliency Detection Co-saliency Detection Temporal RGBD Saliency Detection Motion , 2018 .

[33]  Linwei Ye,et al.  Saliency Detection for Unconstrained Videos Using Superpixel-Level Graph and Spatiotemporal Propagation , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Kurt Debattista,et al.  Advanced High Dynamic Range Imaging , 2017 .

[35]  Panos Nasiopoulos,et al.  A learning-based visual saliency fusion model for High Dynamic Range video (LBVS-HDR) , 2015, 2015 23rd European Signal Processing Conference (EUSIPCO).

[36]  Zhou Wang,et al.  Video saliency incorporating spatiotemporal cues and uncertainty weighting , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[37]  Rita Cucchiara,et al.  A deep multi-level network for saliency prediction , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[38]  Jun Fu,et al.  Dual Attention Network for Scene Segmentation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Wolfgang Heidrich,et al.  High dynamic range display systems , 2004, ACM Trans. Graph..