Point, Line Segment, and Region-Based Stereo Matching for Mobile Robotics

At the heart of every stereo vision algorithm is a solution to the matching problem the problem of finding points in the right and left image that correspond to a single point in the real world. Applying assumptions regarding the epipolar rectification and color similarity between two frames is often not possible for real-world image capture systems, like those used rescue robots. More flexible and robust feature descriptors are necessary to operate under harsh real world conditions. This paper compares the accuracy of disparity images generated using local features including points, line segments, and regions, as well as a global framework implemented using loopy belief propagation. This paper will introduce two new algorithms for stereo matching using line segments and regions, as well as several support structures that optimize the algorithms performance and accuracy. Since few complete frameworks exist for line segment and region features, new algorithms that were developed during the research for this paper will be outlined and evaluated. The comparison includes quantitative evaluation using the Middlebury stereo image pairs and qualitative evaluation using images from a less structured environment. Since this evaluation is grounded in practical environments, processing time is a significant constraint which will be evaluated for each algorithm. This paper will show that line segment-based stereo vision with a gradient descriptor achieves at least a 10% better accuracy than all other methods used in this evaluation while maintaining the low runtime associated with local feature based stereo vision.

[1]  Christopher Joseph Pal,et al.  Learning Conditional Random Fields for Stereo , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  David G. Lowe,et al.  Shape indexing using approximate nearest-neighbour search in high-dimensional spaces , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[4]  John Anderson,et al.  A Region-Based Approach to Stereo Matching for USAR , 2005, RoboCup.

[5]  R.R. Murphy,et al.  Human-robot interaction in USAR technical search: two heads are better than one , 2004, RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No.04TH8759).

[6]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  J.P. Havlicek,et al.  Discrete wavelet transform with optimal joint localization for determining the number of image texture segments , 2002, Proceedings. International Conference on Image Processing.

[8]  Robin R. Murphy,et al.  Human-robot interactions during the robot-assisted urban search and rescue response at the World Trade Center , 2003, IEEE Trans. Syst. Man Cybern. Part B.

[9]  Madasu Hanmandlu,et al.  A fuzzy approach to texture segmentation , 2004, International Conference on Information Technology: Coding and Computing, 2004. Proceedings. ITCC 2004..

[10]  John F. Canny.  A Variational Approach to Edge Detection , 1983, AAAI.

[11]  Ilaria Bartolini,et al.  Windsurf: region-based image retrieval using wavelets , 1999, Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99.

[12]  Sudeep Sarkar,et al.  Robust Visual Method for Assessing the Relative Performance of Edge-Detection Algorithms , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Joseph Kittler,et al.  Robust Line-Segment Extraction using Genetic Algorithms , 1997 .

[14]  Don Ray Murray,et al.  Stereo vision based mapping and navigation for mobile robots , 1997, Proceedings of International Conference on Robotics and Automation.

[15]  W. van der Mark,et al.  A comparative study of fast dense stereo vision algorithms , 2004, IEEE Intelligent Vehicles Symposium, 2004.

[16]  William T. Freeman,et al.  Understanding belief propagation and its generalizations , 2003 .

[17]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[18]  John Anderson,et al.  Flexible binary space partitioning for robotic rescue , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[19]  James J. Little,et al.  Vision-based global localization and mapping for mobile robots , 2005, IEEE Transactions on Robotics.

[20]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[21]  Olga Veksler,et al.  Semi-dense stereo correspondence with dense features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[22]  Hichem Sahli,et al.  Multiscale gradient watersheds of color images , 2003, IEEE Trans. Image Process..

[23]  Richard Szeliski,et al.  High-accuracy stereo depth maps using structured light , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[24]  Miki Haseyama,et al.  Fast line extraction from digital images using line segments , 2003, Systems and Computers in Japan.

[25]  M. Shibata,et al.  Solution for stereo correspondences on active stereo vision robot , 2004, The 8th IEEE International Workshop on Advanced Motion Control, 2004. AMC '04..

[26]  Ki-Sang Hong,et al.  Fast line segment grouping method for finding globally more favorable line segments , 2002, Pattern Recognit..

[27]  O. Basir,et al.  Proximity measure image based region merging for texture segmentation through Gabor filtering and watershed transform , 2003, IEEE International Conference on Robotics, Intelligent Systems and Signal Processing, 2003. Proceedings. 2003.

[28]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[29]  Keun Ho Ryu,et al.  Fast estimation of the number of texture segments using cooccurrence statistics , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[30]  Jacky Baltes,et al.  Practical Region-Based Matching for Stereo Vision , 2004, IWCIA.

[31]  Roberto Cipolla,et al.  Uncalibrated stereo hand-eye coordination , 1994, Image Vis. Comput..

[32]  Reid G. Simmons,et al.  Stereo vision based navigation for Sun-synchronous exploration , 2002, IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Margrit Gelautz,et al.  A layered stereo matching algorithm using image segmentation and global visibility constraints , 2005 .

[35]  P. Fieguth,et al.  A hybrid algorithm using discrete cosine transform and Gabor filter bank for texture segmentation , 2004, Canadian Conference on Electrical and Computer Engineering 2004 (IEEE Cat. No.04CH37513).

[36]  O. Faugeras,et al.  The Geometry of Multiple Images , 1999 .

[37]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[38]  Luc Van Gool,et al.  Wide-baseline stereo matching with line segments , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[39]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[40]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[41]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[42]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[44]  Zhengyou Zhang.  Estimating Motion and Structure from Correspondences of Line Segments between Two Perspective Images , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Ishwar K. Sethi,et al.  Feature Point Correspondence in the Presence of Occlusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..