Advances in Computational Stereo

Extraction of three-dimensional structure of a scene from stereo images is a problem that has been studied by the computer vision community for decades. Early work focused on the fundamentals of image correspondence and stereo geometry. Stereo research has matured significantly throughout the years and many advances in computational stereo continue to be made, allowing stereo to be applied to new and more demanding problems. We review recent advances in computational stereo, focusing primarily on three important topics: correspondence methods, methods for occlusion, and real-time implementations. Throughout, we present tables that summarize and draw distinctions among key ideas and approaches. Where available, we provide comparative analyses and we make suggestions for analyses yet to be done.

[1]  Thomas O. Binford,et al.  Depth from Edge and Intensity Based Stereo , 1981, IJCAI.

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  M. J. McDonnell.  Box-filtering techniques , 1981 .

[4]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[5]  Dana H. Ballard,et al.  Computer Vision , 1982 .

[6]  Martin A. Fischler,et al.  Computational Stereo , 1982, CSUR.

[7]  T. Poggio,et al.  A generalized ordering constraint for stereo correspondence , 1984 .

[8]  A. Gruen.  ADAPTIVE LEAST SQUARES CORRELATION: A POWERFUL IMAGE MATCHING TECHNIQUE , 1985 .

[9]  Takeo Kanade,et al.  Stereo by Intra- and Inter-Scanline Search Using Dynamic Programming , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Jake K. Aggarwal,et al.  Structure from stereo-a review , 1989, IEEE Trans. Syst. Man Cybern..

[11]  Narendra Ahuja,et al.  Surfaces from Stereo: Integrating Feature Matching, Disparity Estimation, and Contour Detection , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  James J. Little,et al.  Direct evidence for occlusion in stereo and motion , 1990, Image Vis. Comput..

[13]  Shankar Chatterjee,et al.  On an analysis of static occlusion in stereo vision , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  André Gagalowicz,et al.  Region based stereo matching oriented image processing , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Jon A. Webb,et al.  The Warp Machine on Navlab , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Richard P. Wildes,et al.  Direct Recovery of Three-Dimensional Scene Geometry From Binocular Stereo Disparity , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Gérard G. Medioni,et al.  3-D Surface Description from Binocular Stereo , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[18]  Jake K. Aggarwal,et al.  Analysis of the stereo correspondence process in scenes with narrow occluding objects , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[19]  Vincent S. Kluth,et al.  Global Least Squares Matching , 1992, [Proceedings] IGARSS '92 International Geoscience and Remote Sensing Symposium.

[20]  Jayant Shah.  A nonlinear diffusion model for discontinuous disparity and half-occlusions in stereo , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[21]  H. K. Nishihara.  Real-time stereo- and motion-based figure ground discrimination and tracking using LOG sign correlation , 1993, Proceedings of 27th Asilomar Conference on Signals, Systems and Computers.

[22]  Andreas Koschan,et al.  What is New in Computational Stereo Since 1989: A Survey on Current Stereo Papers , 1993 .

[23]  Jon A. Webb.  Implementation and performance of fast parallel multi-baseline stereo vision , 1993, 1993 Computer Architectures for Machine Perception.

[24]  Laurent Moll,et al.  Real time correlation-based stereo: algorithm, implementations and applications , 1993 .

[25]  Herve Mathieu.  A Multi-DSP 96002 board , 1993 .

[26]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[28]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Wee-Soon Ching.  A new method of identifying occlusion and specular highlights using active vision , 1994, Proceedings of ICSIPNN '94. International Conference on Speech, Image Processing and Neural Networks.

[30]  Aaron F. Bobick,et al.  Incorporating intensity edges in the recovery of occlusion regions , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[31]  A. Kelly,et al.  Obstacle detection for unmanned ground vehicles: a progress report , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[32]  Takeo Kanade,et al.  CMU Video-Rate Stereo Machine , 1995 .

[33]  Ze-Nian Li,et al.  Depth and occlusion recovery in motion stereo , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[34]  Konstantinos A. Tarabanis,et al.  A survey of sensor planning in computer vision , 1995, IEEE Trans. Robotics Autom..

[35]  Pascal Fua,et al.  Automatic Extraction of Generic House Roofs from High Resolution Aerial Imagery , 1996, ECCV.

[36]  P. Cohen,et al.  Occlusion detection and interpretation based on image reprojection , 1996, Proceeding of Southwest Symposium on Image Analysis and Interpretation.

[37]  Yuichi Ohta,et al.  Occlusion detectable stereo-systematic comparison of detection algorithms , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[38]  Takeo Kanade,et al.  A stereo machine for video-rate dense depth mapping and its new applications , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Richard Szeliski,et al.  Stereo matching with non-linear diffusion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[41]  Roberto Manduchi,et al.  Stereo Without Search , 1996, ECCV.

[42]  Yuichi Ohta,et al.  Occlusion detectable stereo-occlusion patterns in camera matrix , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  Peter I. Corke,et al.  Real-time stereopsis using FPGAs , 1997, FPL.

[44]  John Woodfill,et al.  Real-time stereo vision on the PARTS reconfigurable computer , 1997, Proceedings. The 5th Annual IEEE Symposium on Field-Programmable Custom Computing Machines Cat. No.97TB100186).

[45]  Emanuele Trucco,et al.  Efficient stereo with multiple windowing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[46]  Ruzena Bajcsy,et al.  On occluding contour artifacts in stereo vision , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[47]  Changming Sun.  A Fast Stereo Matching Method , 1997 .

[48]  Charles V. Stewart,et al.  Bias in robust estimation caused by discontinuities and multiple structures , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[50]  Cordelia Schmid,et al.  The Geometry and Matching of Curves in Multiple Views , 1998, ECCV.

[51]  Olivier D. Faugeras,et al.  Variational principles, surface evolution, PDEs, level set methods, and the stereo problem , 1998, IEEE Trans. Image Process..

[52]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[53]  Reinhard Klette,et al.  Computer vision - three-dimensional data from images , 1998 .

[54]  Roberto Manduchi,et al.  Stereo Matching as a Nearest-Neighbor Problem , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Shree K. Nayar,et al.  Ordinal Measures for Image Correspondence , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[56]  Olga Veksler,et al.  Markov random fields with efficient approximations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[57]  M. Strintzis,et al.  Optimized block based disparity estimation in stereo systems using a maximum-flow approach , 1998, Proceedings SIBGRAPI'98. International Symposium on Computer Graphics, Image Processing, and Vision (Cat. No.98EX237).

[58]  Amar Mitiche,et al.  Selective image diffusion: application to disparity estimation , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[59]  Peter K. Allen,et al.  Constraint-based sensor planning for scene modeling , 1999, Proceedings 1999 IEEE International Symposium on Computational Intelligence in Robotics and Automation. CIRA'99 (Cat. No.99EX375).

[60]  Stefan Scherer,et al.  The discriminatory power of ordinal measures - towards a new coefficient , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[61]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[62]  David Beymer,et al.  Real-Time Tracking of Multiple People Using Continuous Detection , 1999 .

[63]  Shigeru Kimura,et al.  A convolver-based real-time stereo machine (SAZAN) , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[64]  Carlo Tomasi,et al.  Multiway cut for stereo and motion with slanted surfaces , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[65]  Aggelos K. Katsaggelos,et al.  Dense Disparity Estimation with a Divide-and-Conquer Disparity Space Image Technique , 1999, IEEE Trans. Multim..

[66]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[67]  Huaibin Zhao.  Global optimal surface from stereo , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[68]  José Santos-Victor,et al.  Intrinsic Images for Dense Stereo Matching with Occlusions , 2000, ECCV.

[69]  Takeo Kanade,et al.  A Cooperative Algorithm for Stereo Matching and Occlusion Detection , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Geoffrey Egnal,et al.  Detecting binocular half-occlusions: empirical comparisons of four approaches , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[71]  Cecilia Verónica Sanz.  Introductory techniques for 3-D computer vision . Emmanuele Trucco - Alessandro Verri , 2000 .

[72]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[73]  Peter I. Corke,et al.  Quantitative Evaluation of Matching Methods and Validity Measures for Stereo Vision , 2001, Int. J. Robotics Res..

[74]  Olivier D. Faugeras,et al.  The geometry of multiple images - the laws that govern the formation of multiple images of a scene and some of their applications , 2001 .

[75]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[76]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[77]  Tomoharu Nakahara,et al.  A Multiple-baseline Stereo Method , 2002 .

[78]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[79]  O. Faugeras,et al.  Variational principles, surface evolution, PDE's, level set methods and the stereo problem , 1998, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[80]  Larry H. Matthies,et al.  Stereo vision for planetary rovers: Stochastic modeling to near real-time implementation , 1991, Optics & Photonics.

[81]  Richard Szeliski,et al.  Stereo Matching with Nonlinear Diffusion , 1998, International Journal of Computer Vision.

[82]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[83]  Zhengyou Zhang,et al.  Determining the Epipolar Geometry and its Uncertainty: A Review , 1998, International Journal of Computer Vision.

[84]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1999, International Journal of Computer Vision.

[85]  Reinhard Männer,et al.  Calculating Dense Disparity Maps from Color Stereo Images, an Efficient Implementation , 2004, International Journal of Computer Vision.

[86]  Peter N. Belhumeur,et al.  A Bayesian approach to binocular steropsis , 1996, International Journal of Computer Vision.

[87]  Trevor Darrell,et al.  Integrated Person Tracking Using Stereo, Color, and Pattern Detection , 2000, International Journal of Computer Vision.

[88]  Kiriakos N. Kutulakos,et al.  A Theory of Shape by Space Carving , 2000, International Journal of Computer Vision.

[89]  Michal Irani,et al.  Computing occluding and transparent motions , 1994, International Journal of Computer Vision.

[90]  Rama Chellappa,et al.  Hierarchical stereo and motion correspondence using feature groupings , 1995, International Journal of Computer Vision.

[91]  Pascal Fua,et al.  Object-centered surface reconstruction: Combining multi-image stereo and shading , 1995, International Journal of Computer Vision.

[92]  Pascal Fua,et al.  A parallel stereo algorithm that produces dense depth maps and preserves image features , 1993, Machine Vision and Applications.