Depth from Refraction Using a Transparent Medium with Unknown Pose and Refractive Index

In this paper, we introduce a novel method for depth acquisition based on refraction of light. A scene is captured directly by a camera and by placing a transparent medium between the scene and the camera. A depth map of the scene is then recovered from the displacements of scene points in the images. Unlike other existing depth from refraction methods, our method does not require prior knowledge of the pose and refractive index of the transparent medium, but instead can recover them directly from the input images. By analyzing the displacements of corresponding scene points in the images, we derive closed form solutions for recovering the pose of the transparent medium and develop an iterative method for estimating the refractive index of the medium. Experimental results on both synthetic and real-world data are presented, which demonstrate the effectiveness of the proposed method.

[1]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[2]  Masatoshi Okutomi,et al.  Reflection Stereo - Novel Monocular Stereo using a Transparent Plate - , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[3]  Alex Pentland,et al.  A New Sense for Depth of Field , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Stephen Lin,et al.  Coded aperture pairs for depth from defocus , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[5]  Leo Grady,et al.  Random Walks for Image Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Hans-Gerd Maas,et al.  New developments in Multimedia Photogrammetry , 2001 .

[7]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[8]  C. Manhart,et al.  Optical 3-D Measurement Techniques , 2007 .

[9]  Sellmeier.  Zur Erklärung der abnormen Farbenfolge im Spectrum einiger Substanzen , 1871 .

[10]  Y. Koike,et al.  Optimum refractive-index profile of the graded-index polymer optical fiber, toward gigabit data links. , 1996, Applied optics.

[11]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[12]  Kiriakos N. Kutulakos,et al.  Dynamic Refraction Stereo , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Yasuyuki Matsushita,et al.  Self-calibrating depth from refraction , 2011, 2011 International Conference on Computer Vision.

[14]  Muralidhara Subbarao,et al.  Depth recovery from blurred edges , 1988, Proceedings CVPR '88: The Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Narendra Ahuja,et al.  A Refractive Camera for Acquiring Stereo and Super-resolution Images , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[16]  Antonio Torralba,et al.  SIFT Flow: Dense Correspondence across Scenes and Its Applications , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Murali Subbarao,et al.  Depth from defocus by changing camera aperture: a spatial domain approach , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Masatoshi Okutomi,et al.  Calibration and rectification for reflection stereo , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Masatoshi Okutomi,et al.  Monocular Range Estimation through a Double-Sided Half-Mirror Plate , 2007, Fourth Canadian Conference on Computer and Robot Vision (CRV '07).

[20]  Narendra Ahuja,et al.  Single camera stereo using planar parallel plate , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[21]  H. Opower.  Multiple view geometry in computer vision , 2002 .

[22]  Shree K. Nayar,et al.  Depth from Diffusion , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  C. Cox,et al.  Measuring the two-dimensional structure of a wavy water surface optically: A surface gradient detector , 1994 .

[24]  David J. Kriegman,et al.  On Refractive Optical Flow , 2004, ECCV.

[25]  Hiroshi Murase,et al.  Surface Shape Reconstruction of a Nonrigid Transport Object Using Refraction and Motion , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[27]  In-So Kweon,et al.  A novel stereo camera system by a biprism , 2000, IEEE Trans. Robotics Autom..