Dual-focus stereo imaging

We present a novel stereo imaging technique called dual-focus stereo imaging or DFSI. DFSI uses a pair of images captured from different viewpoints and at different foci, but with identical wide aperture size. Each image in an DFSI pair exhibits different defocus blur, and the two images form a defocused stereo pair. To model defocus blur, we introduce a defocus kernel map (DKM) that computes the size of the blur disk at each pixel. We derive a novel disparity defocus constraint for computing the DKM in DFSI, and integrate DKM estimation with disparity map estimation to simultaneously recover both maps. We show that the recovered DKMs provide useful guidance for segmenting the in-focus regions. We demonstrate using DFSI in a variety of imaging applications, including low-light imaging, automatic defocus matting, and multifocus photomontage.

[1]  R. Sibson,et al.  A brief description of natural neighbor interpolation , 1981 .

[2]  Michael Potmesil,et al.  A lens and aperture camera model for synthetic image generation , 1981, SIGGRAPH '81.

[3]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Alex Pentland,et al.  A New Sense for Depth of Field , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Xinhua Zhuang,et al.  Image Analysis Using Mathematical Morphology , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Steven A. Shafer,et al.  Depth from focusing and defocusing , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[7]  A. Lynn Abbott,et al.  Active Stereo: Integrating Disparity, Vergence, Focus, Aperture and Calibration for Surface Estimation , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Eero P. Simoncelli,et al.  Range estimation by optical differentiation. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[9]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[10]  Reinhard Koch,et al.  A simple and efficient rectification method for general motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[11]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[12]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[14]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[16]  David Salesin,et al.  Interactive digital photomontage , 2004, ACM Trans. Graph..

[17]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[18]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[19]  Shree K. Nayar,et al.  Rational Filters for Passive Depth from Defocus , 1998, International Journal of Computer Vision.

[20]  Subhasis Chaudhuri,et al.  Depth estimation and image restoration using defocused stereo pairs , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Ramesh Raskar,et al.  Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging , 2004, ACM Trans. Graph..

[22]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..

[23]  Frédo Durand,et al.  Defocus video matting , 2005, ACM Trans. Graph..

[24]  P. Hanrahan,et al.  Light Field Photography with a Hand-held Plenoptic Camera , 2005 .

[25]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[26]  Amit K. Agrawal,et al.  Coded exposure photography: motion deblurring using fluttered shutter , 2006, ACM Trans. Graph..

[27]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.

[28]  S. Nayar,et al.  Projection defocus analysis for scene capture and image display , 2006, ACM Trans. Graph..

[29]  Wojciech Matusik,et al.  Natural video matting using camera arrays , 2006, SIGGRAPH '06.

[30]  Christian Frese,et al.  Robust Depth Estimation by Fusion of Stereo and Focus Series Acquired with a Camera Array , 2006, 2006 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[31]  Michael F. Cohen,et al.  Image and Video Matting: A Survey , 2007, Found. Trends Comput. Graph. Vis..

[32]  Jangheon Kim,et al.  Confocal Disparity Estimation and Recovery of Pinhole Image for Real-Aperture Stereo Camera Systems , 2007, 2007 IEEE International Conference on Image Processing.

[33]  Frédo Durand,et al.  Multi-aperture photography , 2007, ACM Trans. Graph..

[34]  Frédo Durand,et al.  Image and depth from a conventional camera with a coded aperture , 2007, SIGGRAPH 2007.

[35]  Michael F. Cohen,et al.  Optimized Color Sampling for Robust Matting , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Ramesh Raskar,et al.  Dappled photography: mask enhanced cameras for heterodyned light fields and coded aperture refocusing , 2007, ACM Trans. Graph..

[37]  Kiriakos N. Kutulakos,et al.  Confocal Stereo , 2006, International Journal of Computer Vision.

[38]  Jiaya Jia,et al.  High-quality motion deblurring from a single image , 2008, ACM Trans. Graph..

[39]  Bing-Yu Chen,et al.  Extracting depth and matte using a color-filtered aperture , 2008, SIGGRAPH Asia '08.

[40]  B. Hayes Computational Photography , 2008 .

[41]  Shree K. Nayar,et al.  Multiple view image denoising , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Ying Wu,et al.  Removing partial blur in a single image , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Sungkil Lee,et al.  Real-Time Depth-of-Field Rendering Using Anisotropically Filtered Mipmap Interpolation , 2009, IEEE Transactions on Visualization and Computer Graphics.

[44]  Sundaresh Ram,et al.  Removing Camera Shake from a Single Photograph , 2009 .

[45]  S. Nayar,et al.  What are good apertures for defocus deblurring? , 2009, 2009 IEEE International Conference on Computational Photography (ICCP).

[46]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[47]  Kiriakos N. Kutulakos,et al.  Light-Efficient Photography , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.