Real-Time Neighborhood Based Disparity Estimation Incorporating Temporal Evidence

This paper presents a system for dense area based disparity estimation from binocular rectified image sequences with the integration of temporal evidence. The system is using dense 2D optical flow fields and timely displaced disparity estimates to reason about the observed 3D scene flow. This scene flow is then exploited to strengthen timely consistent observations in the disparity estimation. Moreover a novel neighborhood assumption is presented, which allows to seamlessly implement the presented algorithm on the GPU. It is shown that by means of the presented approach the sensitivity to noise and ambiguities observed with plain real-time disparity estimations can be improved, even in fully dynamic scenarios with simultaneous movement of objects and cameras.

[1]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[2]  Changming Sun,et al.  An energy minimisation approach to stereo-temporal dense reconstruction , 2004, ICPR 2004.

[3]  Richard Szeliski,et al.  Stereo matching with non-linear diffusion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  Ruigang Yang,et al.  Multi-resolution real-time stereo on commodity graphics hardware , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[5]  Minglun Gong.  Enforcing Temporal Consistency in Real-Time Stereo Estimation , 2006, ECCV.

[6]  Frederic Devernay,et al.  A Variational Method for Scene Flow Estimation from Stereo Sequences , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[7]  Richard Szeliski,et al.  Stereo Matching with Nonlinear Diffusion , 1998, International Journal of Computer Vision.

[8]  Antti Oulasvirta,et al.  Computer Vision – ECCV 2006 , 2006, Lecture Notes in Computer Science.

[9]  Takeo Kanade,et al.  Three-dimensional scene flow , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Larry H. Matthies,et al.  Kalman filter-based algorithms for estimating depth from image sequences , 1989, International Journal of Computer Vision.

[11]  Ruigang Yang,et al.  Improved Real-Time Stereo on Commodity Graphics Hardware , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[12]  Ye Zhang,et al.  On 3D scene flow and structure estimation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Rudolf Mester,et al.  Free Space Computation Using Stochastic Occupancy Grids and Dynamic Programming , 2008 .