Fast burst images denoising

This paper presents a fast denoising method that produces a clean image from a burst of noisy images. We accelerate alignment of the images by introducing a lightweight camera motion representation called homography flow. The aligned images are then fused to create a denoised output with rapid per-pixel operations in temporal and spatial domains. To handle scene motion during the capture, a mechanism of selecting consistent pixels for temporal fusion is proposed to "synthesize" a clean, ghost-free image, which can largely reduce the computation of tracking motion between frames. Combined with these efficient solutions, our method runs several orders of magnitude faster than previous work, while the denoising quality is comparable. A smartphone prototype demonstrates that our method is practical and works well on a large variety of real examples.

[1]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[3]  Roberto Manduchi,et al.  Bilateral filtering for gray and color images , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[4]  Paul A. Viola,et al.  Robust Real-time Object Detection , 2001 .

[5]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[7]  Martin J. Wainwright,et al.  Image denoising using scale mixtures of Gaussians in the wavelet domain , 2003, IEEE Trans. Image Process..

[8]  Michael Elad,et al.  Fast and robust multiframe super resolution , 2004, IEEE Transactions on Image Processing.

[9]  John Hart,et al.  ACM Transactions on Graphics , 2004, SIGGRAPH 2004.

[10]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[11]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[12]  Leonard McMillan,et al.  Video enhancement using per-pixel virtual exposures , 2005, ACM Trans. Graph..

[13]  Michael J. Black,et al.  Fields of Experts: a framework for learning image priors , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[14]  Jean-Michel Morel,et al.  A non-local algorithm for image denoising , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Lei Zhang,et al.  Color demosaicking via directional linear minimum mean square-error estimation , 2005, IEEE Transactions on Image Processing.

[16]  Frédo Durand,et al.  A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach , 2006, International Journal of Computer Vision.

[17]  Alessandro Foi,et al.  Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering , 2007, IEEE Transactions on Image Processing.

[18]  Jia Chen,et al.  Spatio-Temporal Markov Random Field for Video Denoising , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Karen O. Egiazarian,et al.  Video denoising by sparse 3D transform-domain collaborative filtering , 2007, 2007 15th European Signal Processing Conference.

[20]  Ming Zhang,et al.  Multiresolution Bilateral Filtering for Image Denoising , 2008, IEEE Transactions on Image Processing.

[21]  Richard Szeliski,et al.  Automatic Estimation and Removal of Noise from a Single Image , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Marius Tico,et al.  Multi-frame image denoising and stabilization , 2008, 2008 16th European Signal Processing Conference.

[23]  Shree K. Nayar,et al.  Multiple view image denoising , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Jia Chen,et al.  Noise brush: interactive high quality image-noise separation , 2009, ACM Trans. Graph..

[25]  Marc Levoy,et al.  Gaussian KD-trees for fast high-dimensional filtering , 2009, ACM Trans. Graph..

[26]  Ce Liu,et al.  Exploring new representations and applications for motion analysis , 2009 .

[27]  Eli Shechtman,et al.  PatchMatch: a randomized correspondence algorithm for structural image editing , 2009, ACM Trans. Graph..

[28]  Yifei Lou,et al.  A note on multi-image denoising , 2009, 2009 International Workshop on Local and Non-Local Approximation in Image Processing.

[29]  Jian-Feng Cai,et al.  Blind motion deblurring using multiple images , 2009, J. Comput. Phys..

[30]  Marius Tico,et al.  Artifact-free High Dynamic Range imaging , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[31]  J. Morel,et al.  Multi image noise estimation and denoising , 2010 .

[32]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[33]  Michael F. Cohen,et al.  Seeing Mt. Rainier: Lucky imaging for multi-image denoising, sharpening, and haze removal , 2010, 2010 IEEE International Conference on Computational Photography (ICCP).

[34]  Erik Reinhard,et al.  High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting , 2010 .

[35]  N. Paragios,et al.  A high-quality video denoising algorithm based on reliable motion estimation , 2010 .

[36]  Anat Levin,et al.  Natural image denoising: Optimality and inherent bounds , 2011, CVPR 2011.

[37]  Yasuyuki Matsushita,et al.  Noise suppression in low-light images through joint denoising and demosaicing , 2011, CVPR 2011.

[38]  Irfan A. Essa,et al.  Calibration-free rolling shutter removal , 2012, 2012 IEEE International Conference on Computational Photography (ICCP).

[39]  Eli Shechtman,et al.  Robust patch-based hdr reconstruction of dynamic scenes , 2012, ACM Trans. Graph..

[40]  Seungyong Lee,et al.  Video deblurring for hand-held cameras using patch-based synthesis , 2012, ACM Trans. Graph..

[41]  David E. Jacobs,et al.  Focal Stack Compositing for Depth of Field Control , 2012 .

[42]  Jie Yang,et al.  Fast Patch-Based Denoising Using Approximated Patch Geodesic Paths , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Eli Shechtman,et al.  Patch-based high dynamic range video , 2013, ACM Trans. Graph..

[44]  Michal Irani,et al.  Separating Signal from Noise Using Patch Recurrence across Scales , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Jian Sun,et al.  Bundled camera paths for video stabilization , 2013, ACM Trans. Graph..

[46]  Miguel Granados,et al.  Automatic noise modeling for ghost-free HDR reconstruction , 2013, ACM Trans. Graph..

[47]  Karen O. Egiazarian,et al.  Nonlocal Transform-Domain Filter for Volumetric Data Denoising and Reconstruction , 2013, IEEE Transactions on Image Processing.

[48]  Lauwerens Kuipers,et al.  Handbook of Mathematics , 2014 .