Multi-label Depth Estimation for Graph Cuts Stereo Problems

We describe here a method to compute the depth of a scene from a set of at least two images taken at known view-points. Our approach is based on an energy formulation of the 3D reconstruction problem which we minimize using a graph-cut approach that computes a local minimum whose energy is comparable (modulo a multiple constant) with the energy of the absolute minimum. As usually done, we treat the input images symmetrically, match pixels using photoconsistency, treat occlusion and visibility problems and consider a spatial regularization term which preserves discontinuities. The details of the graph construction as well as the proof of the correctness of the method are given. Moreover we introduce a multi-label refinement algorithm in order to increase the number of depth labels without significantly increasing the computational complexity. Finally we compared our algorithm with the results available in the Middlebury database.

[1]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[2]  Klaus Diepold,et al.  Fast Adaptive Graph-Cuts Based Stereo Matching , 2007, ACIVS.

[3]  Thomas Brox,et al.  High Accuracy Optical Flow Estimation Based on a Theory for Warping , 2004, ECCV.

[4]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[6]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[7]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[8]  Davi Geiger,et al.  Occlusions, Discontinuities, and Epipolar Lines in Stereo , 1998, ECCV.

[9]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Qican Zhang,et al.  Local stereo matching with adaptive support-weight, rank transform and disparity calibration , 2008, Pattern Recognit. Lett..

[12]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[13]  Zheng Zhi.  A Region Based Stereo Matching Algorithm Using Cooperative Optimization , 2009 .

[14]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Sing Bing Kang,et al.  Stereo for Image-Based Rendering using Image Over-Segmentation , 2007, International Journal of Computer Vision.

[16]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[17]  Vladimir Kolmogorov,et al.  Multi-camera Scene Reconstruction via Graph Cuts , 2002, ECCV.

[18]  Vladimir Kolmogorov,et al.  What energy functions can be minimized via graph cuts? , 2002, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Yuichi Taguchi,et al.  Stereo reconstruction with mixed pixels using adaptive over-segmentation , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Patrick Pérez,et al.  Hierarchical Estimation and Segmentation of Dense Motion Fields , 2002, International Journal of Computer Vision.

[21]  Li Xu,et al.  Stereo Matching: An Outlier Confidence Approach , 2008, ECCV.

[22]  Hiroshi Ishikawa,et al.  Exact Optimization for Markov Random Fields with Convex Priors , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  P. Anandan,et al.  Hierarchical Model-Based Motion Estimation , 1992, ECCV.

[24]  Richard Szeliski,et al.  Extracting View-Dependent Depth Maps from a Collection of Images , 2004, International Journal of Computer Vision.

[25]  Olga Veksler,et al.  Markov random fields with efficient approximations , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[26]  Pascal Fua,et al.  A parallel stereo algorithm that produces dense depth maps and preserves image features , 1993, Machine Vision and Applications.

[27]  D. R. Fulkerson,et al.  Maximal Flow Through a Network , 1956 .

[28]  D. Nistér,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Mário A. T. Figueiredo,et al.  Generalized Multi-Camera Scene Reconstruction Using Graph Cuts , 2003 .

[31]  Aaron F. Bobick,et al.  Large Occlusion Stereo , 1999, International Journal of Computer Vision.