Spacetime stereo: a unifying framework for depth from triangulation

Depth from triangulation has traditionally been investigated in a number of independent threads of research, with methods such as stereo, laser scanning, and coded structured light considered separately. We propose a common framework called spacetime stereo that unifies and generalizes many of these previous methods. To show the practical utility of the framework, we develop two new algorithms for depth estimation: depth from unstructured illumination change and depth estimation in dynamic scenes. Based on our analysis, we show that methods derived from the spacetime stereo framework can be used to recover depth in situations in which existing methods perform poorly.

[1]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[2]  Pietro Perona,et al.  3D photography on your desk , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Takeo Kanade,et al.  When Is the Shape of a Scene Unique Given Its Light-Field: A Fundamental Theorem of 3D Vision? , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Michael J. Black,et al.  A framework for the robust estimation of optical flow , 1993, 1993 (4th) International Conference on Computer Vision.

[5]  Szymon Rusinkiewicz,et al.  Stripe boundary codes for real-time structured-light range scanning of moving objects , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[6]  Timothy C. Strand.  Optical Three-Dimensional Sensing For Machine Vision , 1985 .

[7]  Takeo Kanade,et al.  A multibaseline stereo system with active illumination and real-time image acquisition , 1995, Proceedings of IEEE International Conference on Computer Vision.

[8]  Li Zhang,et al.  Rapid shape acquisition using color structured light and multi-pass dynamic programming , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[9]  Holly E. Rushmeier,et al.  High-Quality Texture Reconstruction from Multiple Scans , 2001, IEEE Trans. Vis. Comput. Graph..

[10]  Ray A. Jarvis,et al.  A Perspective on Range Finding Techniques for Computer Vision , 1983, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Linda G. Shapiro,et al.  Acquisition and visualization of colored 3D objects , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[12]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Giancarlo Ferrigno,et al.  Autoscan: A Flexible and Portable 3D Scanner , 1998, IEEE Computer Graphics and Applications.

[14]  Richard Szeliski,et al.  Stereo matching with non-linear diffusion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Kim L. Boyer,et al.  Color-Encoded Structured Light for Rapid Active Ranging , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Takeo Kanade,et al.  A locally adaptive window for signal matching , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[17]  Robert J. Woodham,et al.  Photometric method for determining surface orientation from multiple images , 1980 .

[18]  Steven M. Seitz,et al.  The Space of All Stereo Images , 2004, International Journal of Computer Vision.

[19]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[20]  Brian Curless,et al.  Overview of Active Vision Techniques , 2000 .

[21]  Joaquim Salvi,et al.  Recent progress in coded structured light as a technique to solve the correspondence problem: a survey , 1998, Pattern Recognit..

[22]  Jeffrey A. Jalkio,et al.  Three dimensional inspection using multistripe structured light , 1985 .

[23]  Denis Laurendeau,et al.  3-D Sensing for industrial computer vision , 1988 .

[24]  Andreas F. Koschan,et al.  Color stereo vision using hierarchical block matching and active color illumination , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[25]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[27]  Takeo Kanade,et al.  A very fast VLSI rangefinder , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[28]  Marc Levoy,et al.  Real-time 3D model acquisition , 2002, ACM Trans. Graph..

[29]  Ja-Ling Wu,et al.  Range data acquisition using color structured lighting and stereo vision , 1997, Image Vis. Comput..

[30]  David J. Kriegman,et al.  Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction , 2002, International Journal of Computer Vision.

[31]  Philippe Saint-Marc,et al.  A versatile PC-based range finding system , 1991, IEEE Trans. Robotics Autom..

[32]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[33]  Li Zhang,et al.  Spacetime stereo: shape recovery for dynamic scenes , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[34]  Xing Chen,et al.  A laser range scanner designed for minimum calibration complexity , 2001, Proceedings Third International Conference on 3-D Digital Imaging and Modeling.

[35]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[36]  Paul J. Besl,et al.  Active, optical range imaging sensors , 1988, Machine Vision and Applications.

[37]  Marc Levoy,et al.  Better optical triangulation through spacetime analysis , 1995, Proceedings of IEEE International Conference on Computer Vision.

[38]  Szymon Rusinkiewicz,et al.  Spacetime Stereo: A Unifying Framework for Depth from Triangulation , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Yaron Caspi,et al.  Increasing Space-Time Resolution in Video , 2002, ECCV.

[40]  Yukio Sato,et al.  High Speed Rangefinder , 1988, Other Conferences.