Temporally consistent depth map estimation based on 3D-MRF

Mutiview plus associated depth information form a typical 3D video representation. Depth map of each frame in a video sequence is usually estimated by stereo matching approaches separately. As a result it has weak temporal consistency. In this paper, we propose a novel framework based on spatio-temporal Markov Random Fields. It enforces temporal correlation by employing additional state in the graphical model. Improved belief propagation-sequential algorithm is exploited as an efficient optimization scheme to minimize the energy function. The experimental results demonstrate that the proposed method produces dependable depth maps in both spatial and temporal domain.

[1]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[2]  M. Gelautz,et al.  Temporally consistent disparity maps from uncalibrated stereo videos , 2009, 2009 Proceedings of 6th International Symposium on Image and Signal Processing and Analysis.

[3]  Ming Zhang,et al.  Fast stereo matching algorithm based on adaptive window , 2010, 2010 International Conference on Audio, Language and Image Processing.

[4]  Hujun Bao,et al.  Consistent Depth Maps Recovery from a Video Sequence , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Karen O. Egiazarian,et al.  A memory-efficient and time-consistent filtering of depth map sequences , 2010, Electronic Imaging.

[6]  Pedro F. Felzenszwalb,et al.  Efficient belief propagation for early vision , 2004, CVPR 2004.

[7]  Donald Geman,et al.  Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images , 1984 .

[8]  Vladimir Kolmogorov,et al.  Convergent Tree-Reweighted Message Passing for Energy Minimization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[10]  William T. Freeman,et al.  On the optimality of solutions of the max-product belief-propagation algorithm in arbitrary graphs , 2001, IEEE Trans. Inf. Theory.

[11]  Ming Zhang,et al.  MRF-Based Stereo Correspondence and Virtual View Interpolation , 2009, 2009 Third International Symposium on Intelligent Information Technology Application.

[12]  Hai Tao,et al.  Dynamic depth recovery from multiple synchronized video streams , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Stan Z. Li.  Markov Random Field Modeling in Image Analysis , 2009, Advances in Pattern Recognition.

[14]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Robert T. Collins,et al.  Belief Propagation in a 3D Spatio-temporal MRF for Moving Object Detection , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Tomaso Poggio,et al.  Probabilistic Solution of Ill-Posed Problems in Computational Vision , 1987 .

[17]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[18]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[19]  Tomaso Poggio,et al.  Computational vision and regularization theory , 1985, Nature.

[20]  Gareth Funka-Lea,et al.  Graph Cuts and Efficient N-D Image Segmentation , 2006, International Journal of Computer Vision.