Rotationally invariant sparse patch matching on GPU and FPGA

Vector and data-flow processors are particularly strong at dense, regular computation. Sparse, irregular data layouts cause problems because their unpredictable data access patterns prevent computational pipelines from filling effectively. A number of algorithms in image processing have been proposed which are not dense, and instead apply local neighborhood operations to a sparse, irregular set of points. Sparse and irregular data transfer is difficult for modern processors because they have more processing power than memory bandwidth. However, if the computation can be expanded while not increasing the bandwidth, modern processors can be made more efficient. The application targeted in this paper is patch matching over large scenes. Given two sequential frames of video data, corresponding points between the two frames are found. Correspondences are determined by comparing small image patches around each point. By rotating and comparing patches of the image over a range of angles, it is possible to more accurately match them through the scene. Rotation and interpolation are required to produce an appropriate image to compare against. Results for CPU, FPGA, and GPU are presented, with FPGA far outperforming the GPU or CPU due to its potential for high levels of hardware parallelism as the total volume of computation increases.