An FPGA-oriented Graph Cut Algorithm for Accelerating Stereo Vision

Stereo vision, which reconstructs 3-D information from images obtained by two cameras, is one of the topics most actively addressed in the field of computer vision. While the stereo vision problem can be formulated and solved as an energy minimization problem such as a graph cut, the embedded implementation for real-time systems is difficult due to high computational intensity. This paper proposes a novel parallel-processing-friendly graph cut algorithm and its FPGA implementation for accelerating stereo vision, in which object surfaces are estimated by solving a min-cut problem of a 3-D grid graph. In the proposed algorithm, node-wise parallelism is actively extracted by introducing a concept of wave propagation. The implementation experiments reveal that a system for 12 x 12 x 7-node graphs can be implemented on a single FPGA and works at the clock frequency of 100 MHz. In this case, the system achieves 166 times speedup compared to CPU execution of a common software library of a graph cut.

[1]  Tsutomu Maruyama,et al.  An acceleration of a graph cut segmentation with FPGA , 2012, 22nd International Conference on Field Programmable Logic and Applications (FPL).

[2]  Hailin Jin,et al.  Stereo matching with nonparametric smoothness priors in feature space , 2009, CVPR.

[3]  Andrew W. Fitzgibbon,et al.  Global stereo reconstruction under second order smoothness priors , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Peter Elias,et al.  A note on the maximum flow through a network , 1956, IRE Trans. Inf. Theory.

[6]  Richard M. Karp,et al.  Theoretical Improvements in Algorithmic Efficiency for Network Flow Problems , 1972, Combinatorial Optimization.

[7]  P. J. Narayanan,et al.  CUDA cuts: Fast graph cuts on the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Yuichiro Shibata,et al.  FPGA Implementation of A Graph Cut Algorithm For Stereo Vision , 2017, HEART.

[9]  Yuichiro Shibata,et al.  A new stereo formulation not using pixel and disparity models , 2018, ArXiv.