Computer vision signal processing on graphics processing units

This paper shows speedups attained by using computer graphics hardware for implementation of computer vision algorithms by efficiently mapping mathematical operations of computer vision onto modem computer graphics architecture. As an example computer vision algorithm, we implement a real-time projective camera motion tracking routine on modern, GeForce FX class GPUs (graphics processing units). Algorithms are implemented using OpenGL and the nVIDIA Cg fragment shaders. Trade-offs between computer vision requirements and GPU resources are discussed. Algorithm implementation is examined closely, and hardware bottlenecks are addressed to examine the performance of GPU architecture for computer vision. It is shown that significant speedups can be achieved, while leaving the CPU free for other signal processing tasks. Applications of our work include wearable, computer mediated reality systems that use both computer vision and computer graphics, and require realtime processing with low-latency and high throughput provided by modem GPUs.

[1]  Steve Mann,et al.  Painting with looks: photographic images from video using quantimetric processing , 2002, MULTIMEDIA '02.

[2]  Peter J. Burt.  A pyramid-based front-end processor for dynamic vision applications , 2002 .

[3]  Steve Mann.  Wearable Computing: Toward Humanistic Intelligence , 2001 .

[4]  J. Krüger,et al.  Linear algebra operators for GPU implementation of numerical algorithms , 2003, ACM Trans. Graph..

[5]  Steve Mann,et al.  EyeTap Devices for Augmented, Deliberately Diminished, or Otherwise Altered Visual Perception of Rigid Planar Patches of Real-World Scenes , 2002, Presence: Teleoperators & Virtual Environments.

[6]  W. James MacLean,et al.  Video-rate stereo depth measurement on programmable hardware , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[7]  Pat Hanrahan,et al.  Efficient partitioning of fragment shaders for multipass rendering on programmable graphics hardware , 2002, HWWS '02.

[8]  Masatoshi Ishikawa,et al.  A real-time visual processing system using a general-purpose vision chip , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[9]  William R. Mark,et al.  Cg: a system for programming graphics hardware in a C-like language , 2003, ACM Trans. Graph..

[10]  Thomas Ertl,et al.  Accelerating Morphological Analysis with Graphics Hardware , 2000, VMV.

[11]  Anthony J. Sherbondy,et al.  Fast volume segmentation with simultaneous visualization using programmable graphics hardware , 2003, IEEE Visualization, 2003. VIS 2003..

[12]  Michael D. McCool,et al.  Shader metaprogramming , 2002, HWWS '02.

[13]  Erik Lindholm,et al.  A user-programmable vertex engine , 2001, SIGGRAPH.

[14]  Pat Hanrahan,et al.  A real-time procedural shading system for programmable graphics hardware , 2001, SIGGRAPH.

[15]  Mark Oskin,et al.  Using modern graphics architectures for general-purpose computing: a framework and analysis , 2002, 35th Annual IEEE/ACM International Symposium on Microarchitecture, 2002. (MICRO-35). Proceedings..

[16]  Steve Mann,et al.  Mediated reality using computer graphics hardware for computer vision , 2002, Proceedings. Sixth International Symposium on Wearable Computers,.