Operating Principles of Time-of-Flight Depth Cameras

All current ToF depth camera products in the market are based on the In-Pixel Photo-Mixing Devices technology. This chapter introduces operations of continuous wave homodyne amplitude modulation systems within In-Pixel devices both for sinusoidal and square wave modulation cases. The chapter details various implementation issues affecting the practical operation and performance of ToF depth cameras which readers should be aware of, including flying pixels and multipath interference. The last part of the chapter synthesizes the theory by analyzing the most popular ToF depth camera products, including products from Mesa Imaging, PMD, and Softkinetics, as well as the second generation Kinect.

[1]  Robert Mahony,et al.  Statistical analysis of measurement processes for time-of-flight cameras , 2009, Optical Engineering + Applications.

[2]  Michael J. Cree,et al.  Review of methods for resolving multi-path interference in Time-of-Flight range cameras , 2014, IEEE SENSORS 2014 Proceedings.

[3]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[4]  Michael J. Cree,et al.  Understanding and Ameliorating Mixed Pixels and Multipath Interference in AMCW Lidar , 2013 .

[5]  Tim Weyrich,et al.  Capturing Time-of-Flight data with confidence , 2011, CVPR 2011.

[6]  Ramesh Raskar,et al.  Resolving Multi-path Interference in Time-of-Flight Imaging via Modulation Frequency Diversity and Sparse Regularization , 2014, Optics letters.

[7]  T. Kahlmann,et al.  Calibration and development for increased accuracy of 3D range imaging cameras , 2008 .

[8]  Philip A. Chou,et al.  SPUMIC: Simultaneous phase unwrapping and multipath interference cancellation in time-of-flight cameras using spectral methods , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[9]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[10]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Andrew W. Fitzgibbon,et al.  A rational function lens distortion model for general cameras , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[12]  Peter Seitz,et al.  Robust Optical Time-of-Flight Range Imaging Based on Smart Pixel Structures , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[13]  Michael J. Cree,et al.  Closed-form inverses for the mixed pixel/multipath interference problem in AMCW lidar , 2012, Electronic Imaging.

[14]  Bernd Scholz-Reiter,et al.  Modeling Distance Nonlinearity in ToF Cameras and Correction Based on Integration Time Offsets , 2011, CIARP.

[15]  Jörg Krüger,et al.  Identification and correction of flying pixels in range camera data , 2008, SCCG.

[16]  Michael J. Cree,et al.  Separating true range measurements from multi-path and scattering interference in commercial range cameras , 2011, Electronic Imaging.

[17]  Ramesh Raskar,et al.  Coded time of flight cameras , 2013, ACM Trans. Graph..

[18]  Nick Baker,et al.  Xbox 360 System Architecture , 2006, IEEE Micro.