Stereo Time-of-Flight with Constructive Interference

This paper describes a novel method to acquire depth images using a pair of ToF (Time-of-Flight) cameras. As opposed to approaches that filter, calibrate or do 3D reconstructions posterior to the image acquisition, we combine the measurements of the two cameras within a modified acquisition procedure. The new proposed stereo-ToF acquisition is composed of three stages during which we actively modify the infrared lighting of the scene: first, the two cameras emit an infrared signal one after the other (stages 1 and 2), and then, simultaneously (stage 3). Assuming the scene is static during the three stages, we gather the depth measurements obtained with both cameras and define a cost function to optimize the two depth images. A qualitative and quantitative evaluation of the performance of the proposed stereo-ToF acquisition is provided both for simulated and real ToF cameras. In both cases, the stereo-ToF acquisition produces more accurate depth measurements. Moreover, an extension to the multi-view ToF case and a detailed study on the interference specifications of the system are included.

[1]  Young Min Kim,et al.  Multi-view image and ToF sensor fusion for dense 3D reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[2]  Reinhard Koch,et al.  MixIn3D: 3D Mixed Reality with ToF-Camera , 2009, Dyn3D.

[3]  Gerd Hirzinger,et al.  Extrinsic and depth calibration of ToF-cameras , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Timo Schairer,et al.  Fusion of range and color images for denoising and resolution enhancement with a non-local filter , 2010, Comput. Vis. Image Underst..

[5]  Gabriella Kókai,et al.  Increasing depth lateral resolution based on sensor fusion , 2008, Int. J. Intell. Syst. Technol. Appl..

[6]  Stephan D. Weiss,et al.  Calibration of time-of-flight range imaging cameras , 2008, 2008 15th IEEE International Conference on Image Processing.

[7]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[8]  R. Koch,et al.  CALIBRATION OF A PMD-CAMERA USING A PLANAR CALIBRATION PATTERN TOGETHER WITH A MULTI-CAMERA SETUP , 2008 .

[9]  Sebastian Thrun,et al.  3D shape scanning with a time-of-flight camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Thomas Martinetz,et al.  Shading constraint improves accuracy of time-of-flight measurements , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[11]  Didier Stricker,et al.  Algorithms for 3D Shape Scanning with a Depth Camera , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Reinhard Koch,et al.  A Combined Approach for Estimating Patchlets from PMD Depth Images and Stereo Intensity Images , 2007, DAGM-Symposium.

[13]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Rasmus Larsen,et al.  Fusion of stereo vision and Time-Of-Flight imaging for improved 3D estimation , 2008, Int. J. Intell. Syst. Technol. Appl..

[15]  Ruigang Yang,et al.  Spatial-Temporal Fusion for High Accuracy Depth Maps Using Dynamic MRFs , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Jens T. Thielemann,et al.  Modelling and compensating measurement errors caused by scattering in time-of-flight cameras , 2008, Optical Engineering + Applications.

[17]  Reinhard Koch,et al.  Time‐of‐Flight Cameras in Computer Graphics , 2010, Comput. Graph. Forum.

[18]  Andreas Nüchter,et al.  Robust 3D-mapping with time-of-flight cameras , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Andreas Kolb,et al.  Real-time simulation of time-of-flight sensors , 2009, Simul. Model. Pract. Theory.

[20]  Reinhard Koch,et al.  Time-of-Flight sensor calibration for accurate range sensing , 2010, Comput. Vis. Image Underst..

[21]  Reinhard Koch,et al.  Calibration of focal length and 3D pose based on the reflectance and depth image of a planar object , 2008, Int. J. Intell. Syst. Technol. Appl..