Kinect range sensing: Structured-light versus Time-of-Flight Kinect

This work compares Kinect Structured-Light with Kinect Time-of-Flight cameras.The results offer descriptions under which condition one is superior to the other.Solid insight of the devices is given to make decisions on their application.We propose a set of nine tests for comparing both Kinects, five of which are novel. Recently, the new Kinect One has been issued by Microsoft, providing the next generation of real-time range sensing devices based on the Time-of-Flight (ToF) principle. As the first Kinect version was using a structured light approach, one would expect various differences in the characteristics of the range data delivered by both devices.This paper presents a detailed and in-depth comparison between both devices. In order to conduct the comparison, we propose a framework of seven different experimental setups, which is a generic basis for evaluating range cameras such as Kinect. The experiments have been designed with the goal to capture individual effects of the Kinect devices as isolatedly as possible and in a way, that they can also be adopted, in order to apply them to any other range sensing device. The overall goal of this paper is to provide a solid insight into the pros and cons of either device. Thus, scientists who are interested in using Kinect range sensing cameras in their specific application scenario can directly assess the expected, specific benefits and potential problem of either device.

[1]  Benjamin Langmann,et al.  Scanning 2D/3D monocular camera , 2011, 2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON).

[2]  Timo Kahlmann,et al.  CALIBRATION FOR INCREASED ACCURACY OF THE RANGE IMAGING CAMERA SWISSRANGER , 2006 .

[3]  Andreas Kolb,et al.  Real-Time Motion Artifact Compensation for PMD-ToF Images , 2013, Time-of-Flight and Depth Imaging.

[4]  ZhangZhengyou A Flexible New Technique for Camera Calibration , 2000 .

[5]  P. Payeur,et al.  Calibration of a network of Kinect sensors for robotic inspection over a large workspace , 2013, 2013 IEEE Workshop on Robot Vision (WORV).

[6]  Marcus A. Magnor,et al.  Markerless Motion Capture using multiple Color-Depth Sensors , 2011, VMV.

[7]  Tim Weyrich,et al.  Real-Time 3D Reconstruction in Dynamic Scenes Using Point-Based Fusion , 2013, 2013 International Conference on 3D Vision.

[8]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Young Min Kim,et al.  Multi-view image and ToF sensor fusion for dense 3D reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[10]  Radu Horaud,et al.  Time-of-Flight Cameras: Principles, Methods and Applications , 2012 .

[11]  Jidong Huang,et al.  Study on the use of Microsoft Kinect for robotics applications , 2012, Proceedings of the 2012 IEEE/ION Position, Location and Navigation Symposium.

[12]  Rahul Nair,et al.  A State of the Art Report on Kinect Sensor Setups in Computer Vision , 2013, Time-of-Flight and Depth Imaging.

[13]  Reinhard Koch,et al.  A Comparison of PMD-Cameras and Stereo-Vision for the Task of Surface Reconstruction using Patchlets , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  D. Falie,et al.  Distance errors correction for the Time of Flight (ToF) cameras , 2008, 2008 IEEE International Workshop on Imaging Systems and Techniques.

[15]  Jörg Krüger,et al.  Identification and correction of flying pixels in range camera data , 2008, SCCG.

[16]  Achim J. Lilienthal,et al.  Comparative evaluation of range sensor accuracy for indoor mobile robotics and automated logistics applications , 2013, Robotics Auton. Syst..

[17]  Kiriakos N. Kutulakos,et al.  Transparent and Specular Object Reconstruction , 2010, Comput. Graph. Forum.

[18]  Andreas Kolb,et al.  Compensation of Motion Artifacts for Time-of-Flight Cameras , 2009, Dyn3D.

[19]  Mario Ciampi,et al.  Controller-free exploration of medical image data: Experiencing the Kinect , 2011, 2011 24th International Symposium on Computer-Based Medical Systems (CBMS).

[20]  Heinrich Müller,et al.  Impact of Thermal and Environmental Conditions on the Kinect Sensor , 2012, WDIA.

[21]  Reinhard Koch,et al.  Time‐of‐Flight Cameras in Computer Graphics , 2010, Comput. Graph. Forum.

[22]  Reinhard Koch,et al.  Technical Foundation and Calibration Methods for Time-of-Flight Cameras , 2013, Time-of-Flight and Depth Imaging.

[23]  Klaus-Dieter Kuhnert,et al.  Fusion of Stereo-Camera and PMD-Camera Data for Real-Time Suited Precise 3D Environment Reconstruction , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Reinhard Koch,et al.  Time-of-Flight sensor calibration for accurate range sensing , 2010, Comput. Vis. Image Underst..

[25]  Tomás Pajdla,et al.  3D with Kinect , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[26]  Juho Kannala,et al.  Joint Depth and Color Camera Calibration with Distortion Correction , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Inmaculada Coma,et al.  Augmented Mirror: Interactive Augmented Reality System Based on Kinect , 2011, INTERACT.

[28]  Bernd Jähne,et al.  Efficient and robust reduction of motion artifacts for 3D Time-of-Flight cameras , 2011, 2011 International Conference on 3D Imaging (IC3D).

[29]  Fabio Remondino,et al.  Range imaging technology: new developments and applications for people identification and tracking , 2007, Electronic Imaging.

[30]  Radu Horaud,et al.  Fusion of Range and Stereo Data for High-Resolution Scene-Modeling , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Kiriakos N. Kutulakos,et al.  Transparent and Specular Object Reconstruction , 2010, Comput. Graph. Forum.

[32]  Lena Maier-Hein,et al.  Real-Time Range Imaging in Health Care: A Survey , 2013, Time-of-Flight and Depth Imaging.

[33]  Marcus A. Magnor,et al.  A Survey on Time-of-Flight Stereo Fusion , 2013, Time-of-Flight and Depth Imaging.

[34]  T. Ringbeck,et al.  © Author(s) 2007. This work is licensed under a Creative Commons License. Multidimensional measurement by using 3-D PMD sensors , 2022 .

[35]  Horst Bischof,et al.  A Duality Based Approach for Realtime TV-L1 Optical Flow , 2007, DAGM-Symposium.

[36]  Ramesh Raskar,et al.  Coded time of flight cameras , 2013, ACM Trans. Graph..

[37]  Shahram Izadi,et al.  Modeling Kinect Sensor Noise for Improved 3D Reconstruction and Tracking , 2012, 2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission.

[38]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[39]  Frauke Driewer,et al.  Analysis and characterization of the PMD camera for application in mobile robotics , 2008 .

[40]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[41]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[42]  Matthias Nießner,et al.  Real-time 3D reconstruction at scale using voxel hashing , 2013, ACM Trans. Graph..

[43]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[44]  Andreas Kolb,et al.  Lateral and Depth Calibration of PMD-Distance Sensors , 2006, ISVC.

[45]  Bernd Jähne,et al.  Ground Truth for Evaluating Time of Flight Imaging , 2013, Time-of-Flight and Depth Imaging.

[46]  Otmar Loffeld,et al.  Depth Camera Technology Comparison and Performance Evaluation , 2012, ICPRAM.

[47]  Peter Eisert,et al.  Fast and High Resolution 3D Face Scanning , 2007, 2007 IEEE International Conference on Image Processing.

[48]  Pushmeet Kohli,et al.  When Can We Use KinectFusion for Ground Truth Acquisition , 2012 .

[49]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[50]  Song Zhang,et al.  High-Resolution, Real-time 3D Shape Acquisition , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[51]  Michael J. Cree,et al.  Separating true range measurements from multi-path and scattering interference in commercial range cameras , 2011, Electronic Imaging.

[52]  Achim J. Lilienthal,et al.  Comparative Evaluation of Range Sensor Accuracy in Indoor Environments , 2011, ECMR.

[53]  Sven Behnke,et al.  Multi-frequency Phase Unwrapping for Time-of-Flight cameras , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[54]  David Kim,et al.  Shake'n'sense: reducing interference for overlapping structured light depth cameras , 2012, CHI.