A Combined Approach for Estimating Patchlets from PMD Depth Images and Stereo Intensity Images

Real-time active 3D range cameras based on time-of-flight technology using the Photonic Mixer Device (PMD) can be considered as a complementary technique for stereo-vision based depth estimation. Since those systems directly yield 3D measurements, they can also be used for initializing vision based approaches, especially in highly dynamic environments. Fusion of PMD depth images with passive intensity-based stereo is a promising approach for obtaining reliable surface reconstructions even in weakly textured surface regions. In this work a PMD-stereo fusion algorithm for the estimation of patchlets from a combined PMD-stereo camera rig will be presented. As patchlet we define an oriented small planar 3d patch with associated surface normal. Least-squares estimation schemes for estimating patchlets from PMD range images as well as from a pair of stereo images are derived. It is shown, how those two approaches can be fused into one single estimation, that yields results even if either of the two single approaches fails.

[1]  Andrew Zisserman,et al.  Wide baseline stereo matching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  Donald R. Murray.  Patchlets : a method of interpreting correlation stereo 3D data , 2003 .

[4]  J. Little,et al.  Segmenting correlation stereo range images using surface elements , 2004 .

[5]  John Billingsley,et al.  Mechatronics and machine vision in practice , 2008 .

[6]  Klaus-Dieter Kuhnert,et al.  Fusion ofStereo-Camera andPMD-Camera DataforReal- TimeSuited Precise 3DEnvironment Reconstruction , 2006 .

[7]  Bill Triggs,et al.  Detecting Keypoints with Stable Position, Orientation, and Scale under Illumination Changes , 2004, ECCV.

[8]  Andreas Kolb,et al.  Lateral and Depth Calibration of PMD-Distance Sensors , 2006, ISVC.

[9]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[10]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[11]  Zhigang Zhang,et al.  Untersuchung und Charakterisierung von Photomischdetektor - Strukturen und ihren Grundschaltun - gen , 2006 .

[12]  Reinhard Koch,et al.  Single View Motion Tracking by Depth and Silhouette Information , 2007, SCIA.

[13]  Carlo Tomasi,et al.  3D head tracking based on recognition and interpolation using a time-of-flight depth sensor , 2004, CVPR 2004.

[14]  Timo Kahlmann,et al.  CALIBRATION FOR INCREASED ACCURACY OF THE RANGE IMAGING CAMERA SWISSRANGER , 2006 .

[15]  Tobias Pietzsch,et al.  A Method of Estimating Oriented Surface Elements from Stereo Images , 2005, BMVC.

[16]  Richard Szeliski,et al.  Surface modeling with oriented particle systems , 1992, SIGGRAPH.

[17]  J. Chris McGlone,et al.  Manual of photogrammetry , 2004 .

[18]  Alice Biber,et al.  Time-of-flight range imaging with a custom solid state image sensor , 1999, Industrial Lasers and Inspection.

[19]  Soren W. Henriksen,et al.  Manual of photogrammetry , 1980 .

[20]  J. Little,et al.  Patchlets: a method of interpreting correlation stereo three-dimensional data , 2004 .

[21]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[22]  Reinhard Koch,et al.  Supporting Structure from Motion with a 3D-Range-Camera , 2007, SCIA.

[23]  Ian D. Reid,et al.  Locally Planar Patch Features for Real-Time Structure from Motion , 2004, BMVC.

[24]  Tony DeRose,et al.  Piecewise smooth surface reconstruction , 1994, SIGGRAPH.