Spacetime Stereo: A Unifying Framework for Depth from Triangulation

Depth from triangulation has traditionally been investigated in a number of independent threads of research, with methods such as stereo, laser scanning, and coded structured light considered separately. We propose a common framework called spacetime stereo that unifies and generalizes many of these previous methods. To show the practical utility of the framework, we develop two new algorithms for depth estimation: depth from unstructured illumination change and depth estimation in dynamic scenes. Based on our analysis, we show that methods derived from the spacetime stereo framework can be used to recover depth in situations in which existing methods perform poorly.

[1]  Paul J. Besl,et al.  Active, optical range imaging sensors , 1988, Machine Vision and Applications.

[2]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[3]  Holly E. Rushmeier,et al.  High-Quality Texture Reconstruction from Multiple Scans , 2001, IEEE Trans. Vis. Comput. Graph..

[4]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Jeffrey A. Jalkio,et al.  Three dimensional inspection using multistripe structured light , 1985 .

[6]  Takeo Kanade,et al.  A very fast VLSI rangefinder , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[7]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[8]  Pietro Perona,et al.  3D photography on your desk , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[9]  Timothy C. Strand.  Optical Three-Dimensional Sensing For Machine Vision , 1985 .

[10]  Philippe Saint-Marc,et al.  A versatile PC-based range finding system , 1991, IEEE Trans. Robotics Autom..

[11]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[12]  Takeo Kanade,et al.  A locally adaptive window for signal matching , 2004, International Journal of Computer Vision.

[13]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[14]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Marc Levoy,et al.  Better optical triangulation through spacetime analysis , 1995, Proceedings of IEEE International Conference on Computer Vision.

[16]  Denis Laurendeau,et al.  3-D Sensing for industrial computer vision , 1988 .

[17]  Ja-Ling Wu,et al.  Range data acquisition using color structured lighting and stereo vision , 1997, Image Vis. Comput..

[18]  Takeo Kanade,et al.  A multibaseline stereo system with active illumination and real-time image acquisition , 1995, Proceedings of IEEE International Conference on Computer Vision.

[19]  David J. Kriegman,et al.  Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction , 2002, International Journal of Computer Vision.

[20]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[21]  Takeo Kanade,et al.  When Is the Shape of a Scene Unique Given Its Light-Field: A Fundamental Theorem of 3D Vision? , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Yaron Caspi,et al.  Increasing Space-Time Resolution in Video , 2002, ECCV.

[23]  Joaquim Salvi,et al.  Recent progress in coded structured light as a technique to solve the correspondence problem: a survey , 1998, Pattern Recognit..

[24]  Szymon Rusinkiewicz,et al.  Spacetime stereo: a unifying framework for depth from triangulation , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Yukio Sato,et al.  High Speed Rangefinder , 1988, Other Conferences.

[26]  Kim L. Boyer,et al.  Color-Encoded Structured Light for Rapid Active Ranging , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Xing Chen,et al.  A laser range scanner designed for minimum calibration complexity , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[28]  T. C. Strand.  Optical processing for machine vision , 1988, 1988., IEEE International Symposium on Circuits and Systems.

[29]  Ray A. Jarvis,et al.  A Perspective on Range Finding Techniques for Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Linda G. Shapiro,et al.  Acquisition and visualization of colored 3D objects , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[31]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[32]  Steven M. Seitz,et al.  The Space of All Stereo Images , 2004, International Journal of Computer Vision.

[33]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[34]  Brian Curless,et al.  Overview of Active Vision Techniques , 2000 .

[35]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Giancarlo Ferrigno,et al.  Autoscan: A Flexible and Portable 3D Scanner , 1998, IEEE Computer Graphics and Applications.

[37]  Richard Szeliski,et al.  Stereo matching with non-linear diffusion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[38]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[39]  Andreas F. Koschan,et al.  Color stereo vision using hierarchical block matching and active color illumination , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[40]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.