Extreme Low-Light Environment-Driven Image Denoising over Permanently Shadowed Lunar Regions with a Physical Noise Model

Recently, learning-based approaches have achieved impressive results in the field of low-light image denoising. Some state of the art approaches employ a rich physical model to generate realistic training data. However, the performance of these approaches ultimately depends on the realism of the physical model, and many works only concentrate on everyday photography. In this work we present a denoising approach for extremely low-light images of permanently shadowed regions (PSRs) on the lunar surface, taken by the Narrow Angle Camera on board the Lunar Reconnaissance Orbiter satellite. Our approach extends existing learning-based approaches by combining a physical noise model of the camera with real noise samples and training image scene selection based on 3D ray tracing to generate realistic training data. We also condition our denoising model on the camera’s environmental metadata at the time of image capture (such as the camera’s temperature and age), showing that this improves performance. Our quantitative and qualitative results show that our method strongly outperforms the existing calibration routine for the camera and other baselines. Our results could significantly impact lunar science and exploration, for example by aiding the identification of surface water-ice and reducing uncertainty in rover and human traverse planning into PSRs.

[1]  Radu Timofte,et al.  A Brief Review of Image Denoising Algorithms and Beyond , 2019, Inpainting and Denoising Challenges.

[2]  Lei Zhang,et al.  Weighted Nuclear Norm Minimization with Application to Image Denoising , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Dong-Wook Kim,et al.  NTIRE 2019 Challenge on Real Image Denoising: Methods and Results , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[4]  Wangmeng Zuo,et al.  Toward Convolutional Blind Denoising of Real Photographs , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  J. Salmon,et al.  Poisson noise reduction with non-local PCA , 2012, ICASSP.

[6]  Alexander M. Bronstein,et al.  Deep Convolutional Denoising of Low-Light Images , 2017, ArXiv.

[7]  Lei Zhang,et al.  Sparsity-based image denoising via dictionary learning and structural clustering , 2011, CVPR 2011.

[8]  Frederick R. Chromey To Measure the Sky , 2010 .

[9]  Paul G. Lucey,et al.  Direct evidence of surface exposed water ice in the lunar polar regions , 2018, Proceedings of the National Academy of Sciences.

[10]  OsherStanley,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[11]  Jean-Michel Morel,et al.  A Review of Image Denoising Algorithms, with a New One , 2005, Multiscale Model. Simul..

[12]  Sunil Agrawal,et al.  Image denoising review: From classical to state-of-the-art approaches , 2020, Inf. Fusion.

[13]  Michael Elad,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries , 2006, IEEE Transactions on Image Processing.

[14]  Andrea Vedaldi,et al.  Deep Image Prior , 2017, International Journal of Computer Vision.

[15]  Yu Li,et al.  LIME: Low-Light Image Enhancement via Illumination Map Estimation , 2017, IEEE Transactions on Image Processing.

[16]  Ke Xu,et al.  Learning to Restore Low-Light Images via Decomposition-and-Enhancement , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Thomas Brox,et al.  U-Net: Convolutional Networks for Biomedical Image Segmentation , 2015, MICCAI.

[18]  Prasun Mahanti,et al.  Flight Calibration of the LROC Narrow Angle Camera , 2016 .

[19]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[20]  Feng Liu,et al.  Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries in Wavelet Domain , 2009, 2009 Fifth International Conference on Image and Graphics.

[21]  David E. Smith,et al.  Lunar Reconnaissance Orbiter Overview: The Instrument Suite and Mission , 2007 .

[22]  Ling Shao,et al.  STAR: A Structure and Texture Aware Retinex Model , 2019, IEEE Transactions on Image Processing.

[23]  Jiaolong Yang,et al.  A Physics-Based Noise Formation Model for Extreme Low-Light Raw Denoising , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  H. D. Cheng,et al.  A simple and effective histogram equalization approach to image enhancement , 2004, Digit. Signal Process..

[25]  Stephen Lin,et al.  A High-Quality Denoising Dataset for Smartphone Cameras , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[26]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[27]  Fan Zhang,et al.  Brief review of image denoising techniques , 2019, Visual Computing for Industry, Biomedicine, and Art.

[28]  Jian Yang,et al.  MemNet: A Persistent Memory Network for Image Restoration , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Yue Li,et al.  Improving Extreme Low-Light Image Denoising via Residual Learning , 2019, 2019 IEEE International Conference on Multimedia and Expo (ICME).

[30]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[31]  Jia Xu,et al.  Learning to See in the Dark , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[32]  Florian Jug,et al.  Noise2Void - Learning Denoising From Single Noisy Images , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Erkut Erdem,et al.  Burst Photography for Learning to Enhance Extremely Dark Images , 2020, IEEE Transactions on Image Processing.

[34]  Stefan Roth,et al.  Benchmarking Denoising Algorithms with Real Photographs , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Junping Du,et al.  Low-Light Image Enhancement via a Deep Hybrid Network , 2019, IEEE Transactions on Image Processing.

[36]  Mikhail V. Konnik,et al.  High-level numerical simulations of noise in CCD and CMOS photosensors: review and tutorial , 2014, ArXiv.

[37]  Yide Zhang,et al.  A Poisson-Gaussian Denoising Dataset With Real Fluorescence Microscopy Images , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Jaakko Lehtinen,et al.  Noise2Noise: Learning Image Restoration without Clean Data , 2018, ICML.

[39]  Yu-Bin Yang,et al.  Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections , 2016, NIPS.

[40]  David R. Bull,et al.  Automatic contrast enhancement of low-light images based on local statistics of wavelet coefficients , 2010, 2010 IEEE International Conference on Image Processing.

[41]  Guillermo Sapiro,et al.  Non-local sparse models for image restoration , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[42]  D. Kring,et al.  Using Boulder Tracks as a Tool to Understand the Bearing Capacity of Permanently Shadowed Regions of the Moon , 2020, Journal of Geophysical Research: Planets.

[43]  Padhraic Smyth,et al.  On loss functions which minimize to conditional expected values and posterior proba- bilities , 1993, IEEE Trans. Inf. Theory.

[44]  A. McEwen,et al.  Lunar Reconnaissance Orbiter Camera (LROC) Instrument Overview , 2010 .

[45]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[46]  Joonki Paik,et al.  Low-light image enhancement using variational optimization-based Retinex model , 2017, 2017 IEEE International Conference on Consumer Electronics (ICCE).

[47]  Xiaoli Sun,et al.  The Lunar Orbiter Laser Altimeter (LOLA) on NASA's Lunar Reconnaissance Orbiter (LRO) mission , 2007, 2009 Conference on Lasers and Electro-Optics and 2009 Conference on Quantum electronics and Laser Science Conference.

[48]  Wotao Yin,et al.  An Iterative Regularization Method for Total Variation-Based Image Restoration , 2005, Multiscale Model. Simul..

[49]  Jonathan T. Barron,et al.  Burst photography for high dynamic range and low-light imaging on mobile cameras , 2016, ACM Trans. Graph..

[50]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[51]  Lei Zhang,et al.  Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising , 2016, IEEE Transactions on Image Processing.