A Novel Depth Recovery Approach from Multi-View Stereo Based Focusing

In this paper, we propose a novel depth recovery method from multi-view stereo based focusing. Inspired by the 4D light field theory, we discover the relationship between classical multi-view stereo (MVS) and depth from focus (DFF) methods and concern about different frequency distribution in 2D light field space. Then we propose a way to separate the depth recovery into two steps. At the first stage, we choose some depth candidates using existing multi-view stereo method. At the second phase, the depth from focusing algorithm is employed to determine the final depth. As well known, multi-view stereo and depth from focus need different kinds of input images, which can not be acquired at the same time by using traditional imaging system. We have addressed this issue by using a camera array system and synthetic aperture photography. Both multi-view images and distinct defocus blur images can be captured at the same time. Experimental results have shown that our proposed method can take advantages of MVS and DFF and the recovered depth is better than traditional methods.

[1]  Tae Hee Han,et al.  Fast Normalized Cross-Correlation , 2009, Circuits Syst. Signal Process..

[2]  Markus Niederoest,et al.  Automatic 3D reconstruction and visualization of microscopic objects from a monoscopic multifocus image sequence , 2003 .

[3]  Shree K. Nayar,et al.  Depth from Diffusion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Stephen Lin,et al.  Coded aperture pairs for depth from defocus , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[6]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[7]  Aaron F. Bobick,et al.  Large Occlusion Stereo , 1999, International Journal of Computer Vision.

[8]  Long Quan,et al.  Progressive surface reconstruction from images using a local prior , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[9]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[10]  Homer H. Chen,et al.  Light Field Analysis for Modeling Image Formation , 2011, IEEE Transactions on Image Processing.

[11]  Eric Krotkov,et al.  Focusing , 2004, International Journal of Computer Vision.

[12]  Ren Ng.  Fourier Slice Photography , 2005 .

[13]  Marc Levoy,et al.  High performance imaging using large camera arrays , 2005, SIGGRAPH 2005.

[14]  Mark R. Stevens,et al.  Methods for Volumetric Reconstruction of Visual Scenes , 2004, International Journal of Computer Vision.

[15]  Yoav Y. Schechner,et al.  Depth from Defocus vs. Stereo: How Different Really Are They? , 2004, International Journal of Computer Vision.

[16]  Alex Pentland,et al.  A New Sense for Depth of Field , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Marc Levoy,et al.  Light field rendering , 1996, SIGGRAPH.

[18]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Kiriakos N. Kutulakos,et al.  Confocal Stereo , 2006, International Journal of Computer Vision.

[20]  Shree K. Nayar,et al.  Real-Time Focus Range Sensor , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Samuel W. Hasinoff,et al.  Variable-aperture Photography , 2009 .

[22]  Marc Levoy,et al.  Synthetic Aperture Imaging Using Dense Camera Arrays , 2007 .

[23]  Gui Yun Tian,et al.  A novel depth-from-focus-based measurement system for the reconstruction of surface morphology with depth discontinuity , 2009 .

[24]  Stefano Soatto,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE IEEE TRANSACTION OF PATTERN RECO , 2022 .

[25]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).