Photoconsistent Relative Pose Estimation between a PMD 2D3D-Camera and Multiple Intensity Cameras

Active range cameras based on the Photonic Mixer Device (PMD) allow to capture low-resolution depth images of dynamic scenes at high frame rates. To use such devices together with high resolution optical cameras (e.g. in media production) the relative pose of the cameras with respect to each other has to be determined. This task becomes even more challenging, if the camera is to be moved and the scene is highly dynamic. We will present an efficient algorithm for the estimation of the relative pose between a single 2D3D-camera with respect to several optical cameras. The camera geometry together with an intensity consistency criterion will be used to derive a suitable cost function, which will be optimized using gradient descend. It will be shown, how the gradient of the cost function can be efficiently computed from the gradient images of the high resolution optical cameras. We will show, that the proposed method allows to track and to refine the pose of a moving 2D3D-camera for fully dynamic scenes.

[1]  Reinhard Koch,et al.  A Combined Approach for Estimating Patchlets from PMD Depth Images and Stereo Intensity Images , 2007, DAGM-Symposium.

[2]  Klaus-Dieter Kuhnert,et al.  Fusion of Stereo-Camera and PMD-Camera Data for Real-Time Suited Precise 3D Environment Reconstruction , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[4]  Andreas Kolb,et al.  Sub-pixel data fusion and edge-enhanced distance refinement for 2D/3D images , 2008, Int. J. Intell. Syst. Technol. Appl..

[5]  Stefan May,et al.  Calibration and Registration for Precise Surface Reconstruction with TOF Cameras , 2007 .

[6]  Leszek Wojnar,et al.  Image Analysis , 1998 .

[7]  Reinhard Koch,et al.  Calibration of focal length and 3D pose based on the reflectance and depth image of a planar object , 2008, Int. J. Intell. Syst. Technol. Appl..

[8]  Stefan May,et al.  Calibration and registration for precise surface reconstruction with Time-Of-Flight cameras , 2008, Int. J. Intell. Syst. Technol. Appl..

[9]  R. Koch,et al.  CALIBRATION OF A PMD-CAMERA USING A PLANAR CALIBRATION PATTERN TOGETHER WITH A MULTI-CAMERA SETUP , 2008 .

[10]  Lance Williams,et al.  Casting curved shadows on curved surfaces , 1978, SIGGRAPH.

[11]  Reinhard Koch,et al.  Pose estimation and map building with a Time-Of-Flight-camera for robot navigation , 2008, Int. J. Intell. Syst. Technol. Appl..

[12]  Wolfgang Straßer,et al.  On-the-fly scene acquisition with a handy multi-sensor system , 2008, Int. J. Intell. Syst. Technol. Appl..

[13]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[14]  Marc Alexa,et al.  Combining Time-Of-Flight depth and stereo images without accurate extrinsic calibration , 2008, Int. J. Intell. Syst. Technol. Appl..

[15]  S. E. Ghobadi,et al.  First steps in enhancing 3D vision technique using 2D/3D sensors , 2006 .

[16]  Reinhard Koch,et al.  Supporting Structure from Motion with a 3D-Range-Camera , 2007, SCIA.

[17]  Reinhard Koch,et al.  Real-time Estimation of the Camera Path from a Sequence of Intrinsically Calibrated Pmd Depth Images , 2008 .

[18]  Reinhard Koch,et al.  Pose Estimation and Map Building with a PMD-Camera for Robot Navigation , 2007 .