Time-of-Flight and Structured Light Depth Cameras

This book provides a comprehensive overview of the key technologies and applications related to new cameras that have brought 3D data acquisition to the mass market. It covers both the theoretical principles behind the acquisition devices and the practical implementation aspects of the computer vision algorithms needed for the various applications. Real data examples are used in order to show the performances of the various algorithms. The performance and limitations of the depth camera technology are explored, along with an extensive review of the most effective methods for addressing challenges in common applications. Applications covered in specific detail include scene segmentation, 3D scene reconstruction, human pose estimation and tracking and gesture recognition. This book offers students, practitioners and researchers the tools necessary to explore the potential uses of depth data in light of the expanding number of devices available for sale. It explores the impact of these devices on the rapidly growing field of depth-based computer vision.

[1]  Bill Triggs,et al.  Autocalibration and the absolute quadric , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Jing Wang,et al.  An efficient rectification method for trinocular stereovision , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[3]  Sidney Ray,et al.  Applied photographic optics , 1998 .

[5]  Young Min Kim,et al.  Design and calibration of a multi-view TOF sensor fusion system , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Zhengyou Zhang,et al.  Calibration between depth and color sensors for commodity depth cameras , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[7]  A. L. Lluis-Gomez,et al.  Chromatic aberration correction in RAW domain for image quality enhancement in image sensor processors , 2012, 2012 IEEE 8th International Conference on Intelligent Computer Communication and Processing.

[8]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[9]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[10]  Sebastian Thrun,et al.  Unsupervised Intrinsic Calibration of Depth Sensors via SLAM , 2013, Robotics: Science and Systems.

[11]  Jean Ponce,et al.  Accurate Camera Calibration from Multi-View Stereo and Bundle Adjustment , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Yong Wang,et al.  Using human body gestures as inputs for gaming via depth analysis , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[13]  Moon Gi Kang,et al.  Correction of Axial and Lateral Chromatic Aberration With False Color Filtering , 2013, IEEE Transactions on Image Processing.

[14]  R. Koch,et al.  CALIBRATION OF A PMD-CAMERA USING A PLANAR CALIBRATION PATTERN TOGETHER WITH A MULTI-CAMERA SETUP , 2008 .

[15]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[16]  Richard I. Hartley,et al.  Theory and Practice of Projective Rectification , 1999, International Journal of Computer Vision.

[17]  Xia Liu,et al.  Hand gesture recognition using depth data , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[18]  M. Trobina.  Error Model of a Coded-Light Range Sensor , 2007 .

[19]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[20]  Paul R. Cohen,et al.  Camera Calibration with Distortion Models and Accuracy Evaluation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[22]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Robert Mahony,et al.  Statistical analysis of measurement processes for time-of-flight cameras , 2009, Optical Engineering + Applications.

[24]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[25]  Wolfram Burgard,et al.  Simultaneous calibration, localization, and mapping , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[26]  Lili Wang,et al.  A piecewise correction method for high-resolution image of wide-angle lens , 2013, 2013 6th International Congress on Image and Signal Processing (CISP).

[27]  Andrew W. Fitzgibbon,et al.  A rational function lens distortion model for general cameras , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  Juho Kannala,et al.  A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Olivier D. Faugeras,et al.  A theory of self-calibration of a moving camera , 1992, International Journal of Computer Vision.

[30]  Tomás Svoboda,et al.  A Convenient Multicamera Self-Calibration for Virtual Environments , 2005, Presence: Teleoperators & Virtual Environments.

[31]  Akira Takahashi,et al.  A geometric method for calibration of internal camera parameters of fish-eye lenses , 2007 .

[32]  Robert Lange,et al.  3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology , 2006 .

[33]  Elise Lachat,et al.  Assessment and Calibration of a RGB-D Camera (Kinect v2 Sensor) Towards a Potential Use for Close-Range 3D Modeling , 2015, Remote. Sens..

[34]  Jake K. Aggarwal,et al.  View invariant human action recognition using histograms of 3D joints , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[35]  Radu Horaud,et al.  Projective alignment of range and parallax data , 2011, CVPR 2011.

[36]  Charles T. Loop,et al.  Computing rectifying homographies for stereo vision , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[37]  Sing Bing Kang,et al.  Emerging Topics in Computer Vision , 2004 .

[38]  Weidong Geng,et al.  Accurate Intrinsic Calibration of Depth Camera with Cuboids , 2014, ECCV.

[39]  David Kim,et al.  Shake'n'sense: reducing interference for overlapping structured light depth cameras , 2012, CHI.

[40]  David Nistér,et al.  An efficient solution to the five-point relative pose problem , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[41]  Xiaodong Yang,et al.  Recognizing actions using depth motion maps-based histograms of oriented gradients , 2012, ACM Multimedia.

[42]  Hanseok Ko,et al.  Gesture recognition using depth-based hand tracking for contactless controller application , 2012, 2012 IEEE International Conference on Consumer Electronics (ICCE).

[43]  Henry Fuchs,et al.  Reducing interference between multiple structured light depth sensors using motion , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[44]  Zhengyou Zhang,et al.  Camera calibration with one-dimensional objects , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Richard I. Hartley,et al.  Euclidean Reconstruction from Uncalibrated Views , 1993, Applications of Invariance in Computer Vision.

[46]  Reinhard Koch,et al.  Technical Foundation and Calibration Methods for Time-of-Flight Cameras , 2013, Time-of-Flight and Depth Imaging.

[47]  Reinhard Koch,et al.  Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[48]  Young Min Kim,et al.  Multi-view image and ToF sensor fusion for dense 3D reconstruction , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[49]  A. Verri,et al.  A compact algorithm for rectification of stereo pairs , 2000 .

[50]  Sylvain Paris,et al.  6D hands: markerless hand-tracking for computer aided design , 2011, UIST.

[51]  Janne Heikkilä,et al.  A four-step camera calibration procedure with implicit image correction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[52]  Andreas Kolb,et al.  Calibration of the intensity-related distance error of the PMD TOF-camera , 2007, SPIE Optics East.

[53]  O. Faugeras.  Three-dimensional computer vision: a geometric viewpoint , 1993 .

[54]  Reinhard Koch,et al.  Time-of-Flight sensor calibration for accurate range sensing , 2010, Comput. Vis. Image Underst..

[55]  Richard I. Hartley,et al.  Estimation of Relative Camera Positions for Uncalibrated Cameras , 1992, ECCV.

[56]  Kenichi Kanatani,et al.  Calibration of Ultrawide Fisheye Lens Cameras by Eigenvalue Minimization , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[57]  Tomás Pajdla,et al.  3D with Kinect , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[58]  Peter Seitz,et al.  Robust Optical Time-of-Flight Range Imaging Based on Smart Pixel Structures , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[59]  Xiaodong Yang,et al.  EigenJoints-based action recognition using Naïve-Bayes-Nearest-Neighbor , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[60]  Takafumi Taketomi,et al.  Zoom factor compensation for monocular SLAM , 2015, 2015 IEEE Virtual Reality (VR).

[61]  Stefano Soatto,et al.  A Variational Approach to Problems in Calibration of Multiple Cameras , 2007, IEEE Trans. Pattern Anal. Mach. Intell..

[62]  Paulo R. S. Mendonça,et al.  A simple technique for self-calibration , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).