Real-time semi-global dense stereo solution with improved sub-pixel accuracy

In this work we focus on creating a real-time dense stereo reconstruction system with accurate sub-pixel estimation. We selected the Semi-Global Matching method as the basis of our system due to its high quality and possible real-time implementations. In our solution we use the Census transform as the matching metric because our results show that it can reduce the matching errors for traffic images compared to classical solutions. We also propose several modifications to the original Semi-Global algorithm to improve the sub-pixel accuracy and the execution time. One of these proposals is the reduction in the number of optimization directions without affecting the results. The second modification is a correction of the energy function to reduce the spread of depth values. Besides these improvements, the paper also introduces a new aggregation method used to reduce the spread of sub-pixel values. Finally we propose a new method to generate sub-pixel interpolation functions based on real-world data. The result of these enhancements is a significant improvement in sub-pixel accuracy. The system was implemented and evaluated on a current generation GPU with a running time of 19ms for image having the resolution 512×383.

[1]  Andreas Klaus,et al.  Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[2]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[3]  M. Shimizu,et al.  Precise sub-pixel estimation on area-based matching , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[4]  Oge Marques,et al.  Stereo depth with a Unified Architecture GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[5]  John Iselin Woodfill,et al.  The Tyzx DeepSea G2 Vision System, ATaskable, Embedded Stereo Camera , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[6]  Reinhard Klette,et al.  Inclusion of a Second-Order Prior into Semi-Global Matching , 2009, PSIVT.

[7]  Dariu Gavrila,et al.  Real-time dense stereo for intelligent vehicles , 2006, IEEE Transactions on Intelligent Transportation Systems.

[8]  Ines Ernst,et al.  Mutual Information Based Semi-Global Stereo Matching on the GPU , 2008, ISVC.

[9]  Ruigang Yang,et al.  A Performance Study on Different Cost Aggregation Approaches Used in Real-Time Stereo Matching , 2007, International Journal of Computer Vision.

[10]  Stefan K. Gehrig,et al.  A Real-Time Low-Power Stereo Vision Engine Using Semi-Global Matching , 2009, ICVS.

[11]  Heiko Hirschmüller,et al.  Evaluation of Cost Functions for Stereo Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Masatoshi Okutomi,et al.  Precise Sub-Pixel Estimation on Area-Based Matching , 2001, ICCV.

[13]  Jonathan M. Garibaldi,et al.  Real-Time Correlation-Based Stereo Vision with Reduced Border Errors , 2002, International Journal of Computer Vision.

[14]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.