Multi-camera real-time depth estimation with discontinuity handling on PC graphics hardware

This paper describes a system for dense depth estimation from multiple color images in real-time. Our algorithm runs almost entirely on standard graphics hardware, leaving the main CPU free for other tasks such as image capture and higher level recognition. We follow a plane-sweep approach extended by truncated SSD scores, spatially shiftable windows and best camera selection to handle discontinuities. We do not need specialized hardware and exploit the computational power of freely programmable PC GPU hardware. Dense depth maps are computed with up to 20 fps.

[1]  Christopher Zach,et al.  Optimized Stereo Reconstruction Usina 3D Graphics Hardware , 2003, VMV.

[2]  Richard Szeliski,et al.  Handling occlusions in dense multi-view stereo , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Philippe Colantoni,et al.  Fast and Accurate Color Images Processing Using 3D Graphics Cards , 2003, VMV.

[4]  Robert T. Collins,et al.  A space-sweep approach to true multi-image matching , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Ingemar J. Cox,et al.  A Maximum Likelihood Stereo Algorithm , 1996, Comput. Vis. Image Underst..

[6]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[7]  Ruigang Yang,et al.  Real-time consensus-based scene reconstruction using commodity graphics hardware , 2002, 10th Pacific Conference on Computer Graphics and Applications, 2002. Proceedings..

[8]  Heiko Hirschmüller,et al.  Improvements in real-time correlation-based stereo vision , 2001, CVPR 2001.

[9]  Ruigang Yang,et al.  Multi-resolution real-time stereo on commodity graphics hardware , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..