High Dynamic Range Imaging Using Deep Image Priors

Traditionally, dynamic range enhancement for images has involved a combination of contrast improvement (via gamma correction or histogram equalization) and a denoising operation to reduce the effects of photon noise. More recently, modulo-imaging methods have been introduced for high dynamic range photography to significantly expand dynamic range at the sensing stage itself. The transformation function for both of these problems is highly non-linear, and the image reconstruction procedure is typically non-convex and ill-posed. A popular recent approach is to regularize the above inverse problem via a neural network prior (such as a trained autoencoder), but this requires extensive training over a dataset with thousands of paired regular/HDR image data samples.In this paper, we introduce a new approach for HDR image reconstruction using neural priors that require no training data. Specifically, we employ deep image priors, which have been successfully used for imaging problems such as denoising, super-resolution, inpainting and compressive sensing with promising performance gains over conventional regularization techniques. In this paper, we consider two different approaches to high dynamic range (HDR) imaging – gamma encoding and modulo encoding – and propose a combination of deep image prior and total variation (TV) regularization for reconstructing low-light images. We demonstrate the significant improvement achieved by both of these approaches as compared to traditional dynamic range enhancement techniques.

[1]  Shih-Chia Huang,et al.  Efficient Contrast Enhancement Using Adaptive Gamma Correction With Weighting Distribution , 2013, IEEE Transactions on Image Processing.

[2]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[3]  John D. Austin,et al.  Adaptive histogram equalization and its variations , 1987 .

[4]  Xiaoou Tang,et al.  Image Super-Resolution Using Deep Convolutional Networks , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  E. Land The retinex theory of color vision. , 1977, Scientific American.

[6]  Hemant Tyagi,et al.  On denoising modulo 1 samples of a function , 2018, AISTATS.

[7]  Vladislav Voroninski,et al.  Phase Retrieval Under a Generative Prior , 2018, NeurIPS.

[8]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[9]  Ding Liu,et al.  EnlightenGAN: Deep Light Enhancement Without Paired Supervision , 2019, IEEE Transactions on Image Processing.

[10]  Chun-Liang Li,et al.  One Network to Solve Them All — Solving Linear Inverse Problems Using Deep Projection Models , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[11]  Soumik Sarkar,et al.  LLNet: A deep autoencoder approach to natural low-light image enhancement , 2015, Pattern Recognit..

[12]  Ramesh Raskar,et al.  Unbounded High Dynamic Range Photography Using a Modulo Camera , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[13]  Richard G. Baraniuk,et al.  Exact signal recovery from sparsely corrupted measurements through the Pursuit of Justice , 2009, 2009 Conference Record of the Forty-Third Asilomar Conference on Signals, Systems and Computers.

[14]  Turgay Çelik,et al.  Contextual and Variational Contrast Enhancement , 2011, IEEE Transactions on Image Processing.

[15]  Chinmay Hegde,et al.  Fast, Sample-Efficient Algorithms for Structured Phase Retrieval , 2017, NIPS.

[16]  Michael Elad,et al.  Convolutional Dictionary Learning via Local Processing , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[17]  Chen Wei,et al.  Deep Retinex Decomposition for Low-Light Enhancement , 2018, BMVC.

[18]  Ramesh Raskar,et al.  Unlimited Sampling of Sparse Signals , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[19]  Chinmay Hegde,et al.  Algorithmic Guarantees for Inverse Imaging with Untrained Network Priors , 2019, NeurIPS.

[20]  Reinhard Heckel,et al.  Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks , 2018, ICLR.

[21]  Michael A. Saunders,et al.  Atomic Decomposition by Basis Pursuit , 1998, SIAM J. Sci. Comput..

[22]  Alexandros G. Dimakis,et al.  Compressed Sensing using Generative Models , 2017, ICML.

[23]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[24]  Richard G. Baraniuk,et al.  prDeep: Robust Phase Retrieval with a Flexible Deep Network , 2018, ICML.

[25]  Alexandros G. Dimakis,et al.  Compressed Sensing with Deep Image Prior and Learned Regularization , 2018, ArXiv.

[26]  Chee Seng Chan,et al.  Low-light image enhancement using Gaussian Process for features retrieval , 2019, Signal Process. Image Commun..

[27]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[28]  Chinmay Hegde,et al.  Sample-Efficient Algorithms for Recovering Structured Signals From Magnitude-Only Measurements , 2017, IEEE Transactions on Information Theory.

[29]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[30]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[31]  Yin Zhang,et al.  User’s Guide for TVAL3: TV Minimization by Augmented Lagrangian and Alternating Direction Algorithms , 2010 .

[32]  Karen O. Egiazarian,et al.  Image denoising with block-matching and 3D filtering , 2006, Electronic Imaging.

[33]  Jian Sun,et al.  Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[34]  Timothy Lillicrap,et al.  Deep Compressed Sensing , 2019, ICML.

[35]  Chinmay Hegde,et al.  Signal Reconstruction From Modulo Observations , 2018, 2019 IEEE Global Conference on Signal and Information Processing (GlobalSIP).

[36]  Deanna Needell,et al.  CoSaMP: Iterative signal recovery from incomplete and inaccurate samples , 2008, ArXiv.

[37]  Ali M. Reza,et al.  Realization of the Contrast Limited Adaptive Histogram Equalization (CLAHE) for Real-Time Image Enhancement , 2004, J. VLSI Signal Process..