Stereo Matching on Objects with Fractional Boundary

Conventional stereo matching algorithms assume color constancy on the corresponding opaque pixels in the stereo images. However, when the foreground objects with fractional boundary are blended to the scene behind using unknown alpha values, due to the spatially varying disparities for different layers, the color constancy does not hold any more. In this paper, we address the fractional stereo matching problem. A probability framework is introduced to establish the correspondences of pixel colors, disparities, and alpha values in different layers. We propose an automatic optimization method to solve a Maximum a posteriori (MAP) problem using Expectation-Maximization (EM), given the input of only a narrow-band stereo image pair. Our method naturally encodes pixel occlusion in the formulation of layer blending without a special detection process. We demonstrate the effectiveness of our method using difficult stereo images.

[1]  Michael F. Cohen,et al.  An iterative optimization approach for unified image segmentation and matting , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[2]  Nebojsa Jojic,et al.  Consistent segmentation for optical flow estimation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[3]  Richard Szeliski,et al.  Stereo Matching with Transparency and Matting , 1999, International Journal of Computer Vision.

[4]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[5]  Richard Szeliski,et al.  High-quality video view interpolation using a layered representation , 2004, SIGGRAPH 2004.

[6]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Marisa E. Campbell,et al.  SIGGRAPH 2004 , 2004, INTR.

[8]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[9]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Yi Deng,et al.  Stereo Correspondence with Occlusion Handling in a Symmetric Patch-Based Graph-Cuts Model , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Yi Deng,et al.  A symmetric patch-based correspondence model for occlusion handling , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[12]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Ruigang Yang,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation and Occlusion Handling , 2006, CVPR.

[14]  Dani Lischinski,et al.  A Closed-Form Solution to Natural Image Matting , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Emanuele Trucco,et al.  Rectification with unconstrained stereo geometry , 1997, BMVC.

[16]  Jiaya Jia,et al.  Poisson matting , 2004, SIGGRAPH 2004.

[17]  Wojciech Matusik,et al.  Natural video matting using camera arrays , 2006, SIGGRAPH 2006.

[18]  Andrew W. Fitzgibbon,et al.  Bayesian Estimation of Layers from Multiple Images , 2002, ECCV.

[19]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[20]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  Richard Szeliski,et al.  Stereo matching with reflections and translucency , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[22]  D. Nistér,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Li Hong,et al.  Segment-based stereo matching using graph cuts , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[24]  Wojciech Matusik,et al.  Natural video matting using camera arrays , 2006, SIGGRAPH '06.

[25]  Richard Szeliski,et al.  Boundary Matting for View Synthesis , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[26]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[27]  David Salesin,et al.  A Bayesian approach to digital matting , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.