Accurate Depth-from-Focus Reconstruction using Local and Nonlocal Smoothness Priors

In this paper, we tackle the problem of reconstructing a depth image from a focal stack, which is known as depth-from-focus (DFF) or shape-from-focus (SFF). Recovering a smooth depth image while preserving object structure is a typical issue associated with the conventional DFF or SFF reconstruction techniques. To address this issue, we propose a depth reconstruction method by including two existing local and nonlocal smoothness priors commonly used for natural image matting. Through the combination of local and nonlocal smoothness priors, we can reconstruct a depth image with sharp edges while maintaining spatial consistency. We demonstrate the effectiveness and robustness of the proposed algorithm over synthetic and real scene focal stacks in terms of accuracy and robustness compared to related approaches.

[1]  In So Kweon,et al.  Ring Difference Filter for Fast and Noise Robust Depth From Focus , 2020, IEEE Transactions on Image Processing.

[2]  Tae-Sun Choi,et al.  A novel algorithm for estimation of depth map using image focus for 3D shape recovery in the presence of noise , 2008, Pattern Recognit..

[3]  Peter Corcoran,et al.  Application of preconditioned alternating direction method of multipliers in depth from focal stack , 2017, ArXiv.

[4]  Sheng-Jyh Wang,et al.  Shape-From-Focus Depth Reconstruction With a Spatial Consistency Model , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Michael Möller,et al.  Variational Depth From Focus Reconstruction , 2014, IEEE Transactions on Image Processing.

[6]  Bradley J. Nelson,et al.  Wavelet-based autofocusing and unsupervised segmentation of microscopic images , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[7]  Frank Dellaert,et al.  Structure from motion without correspondence , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[8]  Ze-Nian Li,et al.  Continuous depth map reconstruction from light fields , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[9]  Shree K. Nayar,et al.  Shape from Focus , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  Stefano Soatto,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE IEEE TRANSACTION OF PATTERN RECO , 2022 .

[11]  Soo-Won Kim,et al.  Reduced Energy-Ratio Measure for Robust Autofocusing in Digital Camera , 2009, IEEE Signal Processing Letters.

[12]  Zhongliang Jing,et al.  Evaluation of focus measures in multi-focus image fusion , 2007, Pattern Recognit. Lett..

[13]  Andrea Fusiello,et al.  Generation of All-in-Focus Images by Noise-Robust Selective Fusion of Limited Depth-of-Field Images , 2013, IEEE Transactions on Image Processing.

[14]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Domenec Puig,et al.  Analysis of focus measure operators for shape-from-focus , 2013, Pattern Recognit..

[16]  Bastian Goldlücke,et al.  A Dataset and Evaluation Methodology for Depth Estimation on 4D Light Fields , 2016, ACCV.

[17]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[18]  King-Sun Fu,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Publication Information , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Kiriakos N. Kutulakos,et al.  Confocal Stereo , 2006, International Journal of Computer Vision.

[20]  Mohinder Malhotra Single Image Haze Removal Using Dark Channel Prior , 2016 .

[21]  Yeong-Gil Shin,et al.  Shape-from-focus reconstruction using nonlocal matting Laplacian prior followed by MRF-based refinement , 2020, Pattern Recognit..

[22]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[23]  Haibin Ling,et al.  Saliency Detection on Light Field , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.