A Real-Time Occlusion Aware Hardware Structure for Disparity Map Computation

Many machine vision applications deal with depth estimation in a scene. Disparity map recovery from a stereo image pair has been extensively studied by the computer vision community. Previous methods are mainly restricted to software based techniques on general-purpose architectures, presenting relatively high execution time due to the computationally complex algorithms involved. In this paper a new hardware module suitable for real-time disparity map computation module is realized. This enables a hardware based occlusion-aware parallel-pipelined design, implemented on a single FPGA device with a typical operating frequency of 511 MHz. It provides accurate disparity map computation at a rate of 768 frames per second, given a stereo image pair with a disparity range of 80 pixels and 640x480 pixel spatial resolution. The proposed method allows a fast disparity map computational module to be built, enabling a suitable module for real-time stereo vision applications.

[1]  Sang Uk Lee,et al.  A dense stereo matching using two-pass dynamic programming with generalized ground control points , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Minglun Gong,et al.  Real-Time Stereo Matching Using Orthogonal Reliability-Based Dynamic Programming , 2007, IEEE Transactions on Image Processing.

[4]  Russell Tessier,et al.  Field-Programmable Gate Arrays in Embedded Systems , 2006, EURASIP Journal on Embedded Systems.

[5]  Kurt Konolige,et al.  Small Vision Systems: Hardware and Implementation , 1998 .

[6]  Ruzena Bajcsy,et al.  On occluding contour artifacts in stereo vision , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Shree K. Nayar,et al.  Ordinal Measures for Image Correspondence , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[8]  Peter N. Belhumeur,et al.  A Bayesian approach to binocular steropsis , 1996, International Journal of Computer Vision.

[9]  Laurent Moll,et al.  Real time correlation-based stereo: algorithm, implementations and applications , 1993 .

[10]  Reinhard Männer,et al.  Calculating Dense Disparity Maps from Color Stereo Images, an Efficient Implementation , 2004, International Journal of Computer Vision.

[11]  Carlo Tomasi,et al.  Depth Discontinuities by Pixel-to-Pixel Stereo , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[12]  Pascal Fua,et al.  A parallel stereo algorithm that produces dense depth maps and preserves image features , 1993, Machine Vision and Applications.

[13]  Minglun Gong,et al.  Fast Unambiguous Stereo Matching Using Reliability-Based Dynamic Programming , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Andreas Steininger,et al.  SAD-Based Stereo Matching Using FPGAs , 2009 .

[15]  Shankar Chatterjee,et al.  On an analysis of static occlusion in stereo vision , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Luigi di Stefano,et al.  A fast area-based stereo matching algorithm , 2004, Image Vis. Comput..

[17]  Ioannis Andreadis,et al.  A real-time fuzzy hardware structure for disparity map computation , 2011, Journal of Real-Time Image Processing.

[18]  Olga Veksler.  Extracting dense features for visual correspondence with graph cuts , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  Adam Baumberg,et al.  Reliable feature matching across widely separated views , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[20]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[21]  Marco Platzner,et al.  Field Programmable Logic and Application , 2004, Lecture Notes in Computer Science.

[22]  Q. M. Jonathan Wu,et al.  A New Miniaturized Embedded Stereo-Vision System (MESVS-I) , 2008, 2008 Canadian Conference on Computer and Robot Vision.

[23]  Minglun Gong,et al.  Near real-time reliable stereo matching using programmable graphics hardware , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[24]  Heiko Hirschmüller,et al.  Improvements in real-time correlation-based stereo vision , 2001, CVPR 2001.

[25]  Tsutomu Maruyama,et al.  A Real-Time Stereo Vision System with FPGA , 2003, FPL.

[26]  William B. Thompson,et al.  Disparity Analysis of Images , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  John Iselin Woodfill,et al.  3D Vision: Developing an Embedded Stereo-Vision System , 2007, Computer.

[28]  Alonzo Kelly,et al.  Obstacle detection for unmanned ground vehicles: a progress report , 1995 .

[29]  Alan C. Bovik,et al.  Using chromatic information in edge-based stereo correspondence , 1991, CVGIP Image Underst..

[30]  Georgios Ch. Sirakoulis,et al.  Real-time disparity map computation module , 2008, Microprocess. Microsystems.