Real-time disparity estimation algorithm for stereo camera systems

This paper proposes a real-time stereo matching algorithm using GPU programming. The likelihood model is implemented using GPU programming for real-time operation. And the prior model is proposed to improve the accuracy of disparity estimation. First, the likelihood matching based on rank transform is implemented in GPU programming. The shared memory handling in graphic hardware is introduced in calculating the likelihood model. The prior model considers the smoothness of disparity map and is defined as a pixel-wise energy function using adaptive interaction among neighboring disparities. The disparity is determined by minimizing the joint energy function which combines the likelihood model with prior model. These processes are performed in the multi-resolution approach. The disparity map is interpolated using the reliability of likelihood model and color-based similarity in the neighborhood. This paper evaluates the proposed approach with the Middlebury stereo images. According to the experiments, the proposed algorithm shows good estimation accuracy over 30 frames/second for 640×480 image and 60 disparity range. The proposed disparity estimation algorithm is applied to real-time stereo camera system such as 3-D image display, depth-based object extraction, 3-D rendering, and so on.

[1]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[2]  Minglun Gong,et al.  Real-Time Stereo Matching Using Orthogonal Reliability-Based Dynamic Programming , 2007, IEEE Transactions on Image Processing.

[3]  Sang Uk Lee,et al.  A dense stereo matching using two-pass dynamic programming with generalized ground control points , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[4]  Olga Veksler,et al.  Stereo correspondence by dynamic programming on a tree , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[5]  Richard Szeliski,et al.  Stereo Matching with Nonlinear Diffusion , 1998, International Journal of Computer Vision.

[6]  Miao Liao,et al.  High-Quality Real-Time Stereo Using Adaptive Cost Aggregation and Dynamic Programming , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[7]  Zheng Zhi.  A Region Based Stereo Matching Algorithm Using Cooperative Optimization , 2009 .

[8]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Richard Szeliski,et al.  Stereo matching with non-linear diffusion , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  Minglun Gong,et al.  Near real-time reliable stereo matching using programmable graphics hardware , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Sang Chul Ahn,et al.  Tangible video teleconference system using real-time image-based relighting , 2009, IEEE Transactions on Consumer Electronics.

[12]  D. Nistér,et al.  Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation, and Occlusion Handling , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[14]  Jian Sun,et al.  Symmetric stereo matching for occlusion handling , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Harry Shum,et al.  Stereo computation using radial adaptive windows , 2002, Object recognition supported by user interaction for service robots.

[16]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Minglun Gong,et al.  Fast stereo matching using reliability-based dynamic programming and consistency constraints , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[18]  Liang-Gee Chen,et al.  Hardware-Efficient Belief Propagation , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[19]  Sang Hwa Lee,et al.  MAP-Based Stochastic Diffusion for Stereo Matching and Line Fields Estimation , 2002, International Journal of Computer Vision.

[20]  Sang Uk Lee,et al.  Robust Stereo Matching Using Adaptive Normalized Cross-Correlation , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Olga Veksler,et al.  A Variable Window Approach to Early Vision , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Takeo Kanade,et al.  A Cooperative Algorithm for Stereo Matching and Occlusion Detection , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[23]  Trevor Darrell,et al.  A radial cumulative similarity transform for robust image correspondence , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[24]  Ruigang Yang,et al.  Spatial-Depth Super Resolution for Range Images , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Oscar E. Ruiz,et al.  Realtime Dense Stereo Matching with Dynamic Programming in CUDA , 2009, CEIG.

[26]  Yi Deng,et al.  A Fast Line Segment Based Dense Stereo Algorithm Using Tree Dynamic Programming , 2006, ECCV.

[27]  Narendra Ahuja,et al.  A constant-space belief propagation algorithm for stereo matching , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  J. Biemond,et al.  Correspondence estimation in image pairs , 1999, IEEE Signal Process. Mag..

[29]  Vladimir Kolmogorov,et al.  Computing visual correspondence with occlusions using graph cuts , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[30]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Federico Tombari,et al.  Segmentation-Based Adaptive Support for Accurate Stereo Correspondence , 2007, PSIVT.

[32]  Li Xu,et al.  Stereo Matching: An Outlier Confidence Approach , 2008, ECCV.

[33]  Nanning Zheng,et al.  Stereo Matching Using Belief Propagation , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  M. Bleyer,et al.  Near Real-Time Stereo With Adaptive Support Weight Approaches , 2010 .

[35]  Olga Veksler,et al.  Fast variable window for stereo correspondence using integral images , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[36]  Nam Ik Cho,et al.  Stereo matching using hierarchical belief propagation along ambiguity gradient , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[37]  Olga Veksler,et al.  Stereo Correspondence with Compact Windows via Minimum Ratio Cycle , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Sang Hwa Lee,et al.  Real-time 3D surface modeling for image based relighting , 2009, IEEE Transactions on Consumer Electronics.

[39]  Mohammed Bennamoun,et al.  Reliability analysis of the rank transform for stereo matching , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[40]  Emanuele Trucco,et al.  Efficient stereo with multiple windowing , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Richard Szeliski,et al.  Handling occlusions in dense multi-view stereo , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[42]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[43]  Ze-Nian Li,et al.  Analysis of disparity gradient based cooperative stereo , 1996, IEEE Trans. Image Process..

[44]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.