Exposure Interpolation by Combining Model-driven and Data-driven Methods

Deep learning based methods have penetrated many image processing problems and become dominant solutions to these problems. A natural question raised here is "Is there any space for conventional methods on these problems?" In this paper, exposure interpolation is taken as an example to answer this question and the answer is "Yes". A framework on fusing conventional and deep learning method is introduced to generate an medium exposure image for two large-exposureratio images. Experimental results indicate that the quality of the medium exposure image is increased significantly through using the deep learning method to refine the interpolated image via the conventional method. The conventional method can be adopted to improve the convergence speed of the deep learning method and to reduce the number of samples which is required by the deep learning method.

[1]  Shree K. Nayar,et al.  Determining the Camera Response from Images: What Is Knowable? , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Jan Kautz,et al.  Exposure Fusion , 2007, 15th Pacific Conference on Computer Graphics and Applications (PG'07).

[3]  Jitendra Malik,et al.  Recovering high dynamic range radiance maps from photographs , 1997, SIGGRAPH '08.

[4]  Tomoo Mitsunaga,et al.  Coded rolling shutter photography: Flexible space-time sampling , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[5]  Pradeep Sen,et al.  A versatile HDR video production system , 2011, ACM Trans. Graph..

[6]  Susanto Rahardja,et al.  Hybrid Patching for a Sequence of Differently Exposed Images With Moving Objects , 2013, IEEE Transactions on Image Processing.

[7]  Shiqian Wu,et al.  Selectively Detail-Enhanced Fusion of Differently Exposed Images With Moving Objects , 2014, IEEE Transactions on Image Processing.

[8]  Noah Snavely,et al.  Material recognition in the wild with the Materials in Context Database , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Yoshua Bengio,et al.  Describing Multimedia Content Using Attention-Based Encoder-Decoder Networks , 2015, IEEE Transactions on Multimedia.

[10]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Zhengguo Li,et al.  Multi-scale exposure fusion via gradient domain guided image filtering , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[12]  Yoshihiro Kanamori,et al.  Deep reverse tone mapping , 2017, ACM Trans. Graph..

[13]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Zhengguo Li,et al.  Detail-Enhanced Multi-Scale Exposure Fusion , 2017, IEEE Transactions on Image Processing.

[15]  Christophe De Vleeschouwer,et al.  Single-Scale Fusion: An Effective Approach to Merging Images , 2017, IEEE Trans. Image Process..

[16]  Wei Cao,et al.  Multi-Scale Fusion of Two Large-Exposure-Ratio Images , 2018, IEEE Signal Processing Letters.

[17]  Ling-Yu Duan,et al.  Unified Spatio-Temporal Attention Networks for Action Recognition in Videos , 2019, IEEE Transactions on Multimedia.

[18]  Masahiro Kobayashi,et al.  A 3.4 μm pixel pitch global shutter CMOS image sensor with dual in-pixel charge domain memory , 2019, Japanese Journal of Applied Physics.

[19]  Chi-Wing Fu,et al.  Underexposed Photo Enhancement Using Deep Illumination Estimation , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Qi Wu,et al.  Attend and Imagine: Multi-Label Image Classification With Visual Attention and Recurrent Neural Networks , 2019, IEEE Transactions on Multimedia.

[21]  Shiqian Wu,et al.  Exposure Interpolation Via Hybrid Learning , 2020, ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[22]  Liangcai Zeng,et al.  Hazy Image Decolorization With Color Contrast Restoration , 2020, IEEE Transactions on Image Processing.

[23]  Fahad Shahbaz Khan,et al.  CycleISP: Real Image Restoration via Improved Data Synthesis , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Shiqian Wu,et al.  Single Image Brightening via Multi-Scale Exposure Fusion With Hybrid Learning , 2020, IEEE Transactions on Circuits and Systems for Video Technology.