Coded Aperture Pairs for Depth from Defocus and Defocus Deblurring

The classical approach to depth from defocus (DFD) uses lenses with circular apertures for image capturing. We show in this paper that the use of a circular aperture severely restricts the accuracy of DFD. We derive a criterion for evaluating a pair of apertures with respect to the precision of depth recovery. This criterion is optimized using a genetic algorithm and gradient descent search to arrive at a pair of high resolution apertures. These two coded apertures are found to complement each other in the scene frequencies they preserve. This property enables them to not only recover depth with greater fidelity but also obtain a high quality all-focused image from the two captured images. Extensive simulations as well as experiments on a variety of real scenes demonstrate the benefits of using the coded apertures over conventional circular apertures.

[1]  Stephen Lin,et al.  Coded aperture pairs for depth from defocus , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[2]  Shmuel Peleg,et al.  Two motion-blurred images are better than one , 2005, Pattern Recognit. Lett..

[3]  S. Nayar,et al.  What are good apertures for defocus deblurring? , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[4]  Yoav Y Schechner,et al.  Depth from diffracted rotation. , 2006, Optics letters.

[5]  Chanin Varamit,et al.  Imaging properties of defocused partitioned pupils , 1985 .

[6]  Yoav Y. Schechner,et al.  Depth from Defocus vs. Stereo: How Different Really Are They? , 2004, International Journal of Computer Vision.

[7]  Alex Pentland,et al.  A New Sense for Depth of Field , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Muralidhara Subbarao,et al.  Depth recovery from blurred edges , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[9]  Takashi Matsuyama,et al.  Depth measurement by the multi-focus camera , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[10]  Subhasis Chaudhuri,et al.  A Variational Approach to Recovering Depth From Defocused Images , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  E E Fenimore,et al.  New family of binary arrays for coded aperture imaging. , 1989, Applied optics.

[12]  Edward Raymond Dowski,et al.  Passive Ranging with AN Incoherent Optical System. , 1993 .

[13]  Eero P. Simoncelli,et al.  Range estimation by optical differentiation. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[14]  Murali Subbarao,et al.  Noise sensitivity analysis of depth-from-defocus by a spatial-domain approach , 1997, Optics & Photonics.

[15]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, SIGGRAPH 2007.

[16]  J. Ojeda-Castañeda,et al.  Annular apodizers for low sensitivity to defocus and to spherical aberration. , 1986, Optics letters.

[17]  J. P. Mills,et al.  Effect of aberrations and apodization on the performance of coherent optical systems. II. Imaging , 1986 .

[18]  Stefano Soatto,et al.  Shape and Radiance Estimation from the Information-Divergence of Blurred Images , 2000, ECCV.

[19]  Shree K. Nayar,et al.  Real-Time Focus Range Sensor , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  G. Häusler,et al.  A method to increase the depth of focus by two step image processing , 1972 .

[21]  Kiriakos N. Kutulakos,et al.  Confocal Stereo , 2006, International Journal of Computer Vision.

[22]  Juan Campos,et al.  Axial and Extra-axial Responses in Aberrated Optical Systems with Apodizers. Optimization of the Strehl Ratio , 1989 .

[23]  Wilson S. Geisler,et al.  Maximum-likelihood depth-from-defocus for active vision , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[24]  Deepu Rajan,et al.  Simultaneous Estimation of Super-Resolved Scene and Depth Map from Low Resolution Defocused Observations , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Chi Liu,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, ACM Trans. Graph..

[26]  Steven A. Shafer,et al.  Depth from focusing and defocusing , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[27]  E. Caroli,et al.  Coded aperture imaging in X- and gamma-ray astronomy , 1987 .

[28]  Edward R. Dowski,et al.  Wavefront coding: a modern method of achieving high-performance and/or low-cost imaging systems , 1999, Optics & Photonics.

[29]  Murali Subbarao,et al.  Depth from defocus: A spatial domain approach , 1994, International Journal of Computer Vision.

[30]  Shree K. Nayar,et al.  Rational Filters for Passive Depth from Defocus , 1998, International Journal of Computer Vision.

[31]  Chia-Kai Liang,et al.  Programmable aperture photography: multiplexed light field acquisition , 2008, SIGGRAPH 2008.

[32]  Shree K. Nayar,et al.  Transactions on Pattern Analysis and Machine Intelligence Flexible Depth of Field Photography 1 Depth of Field , 2022 .

[33]  Subhasis Chaudhuri,et al.  Optimal selection of camera parameters for recovery of depth from defocused images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[34]  Mario Bertero,et al.  Regularized deconvolution of multiple images of the same object , 1996 .

[35]  M. Mino,et al.  Improvement in the OTF of a Defocused Optical System Through the Use of Shaded Apertures. , 1971, Applied optics.

[36]  Yoav Y. Schechner,et al.  The optimal axial interval in estimating depth from defocus , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[37]  J Ojeda-Castaneda,et al.  High focal depth by apodization and digital restoration. , 1988, Applied optics.

[38]  Frédo Durand,et al.  4D frequency analysis of computational cameras for depth of field extension , 2009, SIGGRAPH '09.

[39]  J Ojeda-Castañeda,et al.  Bessel annular apodizers: imaging characteristics. , 1987, Applied optics.

[40]  J. H. van Hateren,et al.  Modelling the Power Spectra of Natural Images: Statistics and Information , 1996, Vision Research.

[41]  Stefano Soatto,et al.  A geometric approach to shape from defocus , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  William T. Freeman,et al.  What makes a good model of natural images? , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[44]  W. Welford.  Use of Annular Apertures to Increase Focal Depth , 1960 .

[45]  Subhasis Chaudhuri,et al.  An MRF Model-Based Approach to Simultaneous Recovery of Depth and Restoration from Defocused Images , 1999, IEEE Trans. Pattern Anal. Mach. Intell..