Feature tracking and matching in video using programmable graphics hardware

This paper describes novel implementations of the KLT feature tracking and SIFT feature extraction algorithms that run on the graphics processing unit (GPU) and is suitable for video analysis in real-time vision systems. While significant acceleration over standard CPU implementations is obtained by exploiting parallelism provided by modern programmable graphics hardware, the CPU is freed up to run other computations in parallel. Our GPU-based KLT implementation tracks about a thousand features in real-time at 30 Hz on 1,024 × 768 resolution video which is a 20 times improvement over the CPU. The GPU-based SIFT implementation extracts about 800 features from 640 × 480 video at 10 Hz which is approximately 10 times faster than an optimized CPU implementation.

[1]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[2]  C. Tomasi.  Detection and Tracking of Point Features , 1991 .

[3]  Ruigang Yang,et al.  Fast Image Segmentation and Smoothing Using Commodity Graphics Hardware , 2002, J. Graphics, GPU, & Game Tools.

[4]  Sorin A. Huss,et al.  Real time image processing based on reconfigurable hardware acceleration , 2002 .

[5]  Ruigang Yang,et al.  Multi-resolution real-time stereo on commodity graphics hardware , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[6]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[7]  Horst Bischof,et al.  Hierarchical Disparity Estimation with Programmable 3D Hardware , 2004 .

[8]  Steve Mann,et al.  Computer vision signal processing on graphics processing units , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[9]  Reinhard Koch,et al.  Real-time multi-stereo depth estimation on GPU with approximative discontinuity handling , 2004 .

[10]  Reinhard Koch,et al.  Visual Modeling with a Hand-Held Camera , 2004, International Journal of Computer Vision.

[11]  Bernhard Rinner,et al.  An Embedded Smart Camera on a Scalable Heterogeneous Multi-DSP System , 2004 .

[12]  Martin Rumpf,et al.  Image Registration by a Regularized Gradient Flow. A Streaming Implementation in DX9 Graphics Hardware , 2004, Computing.

[13]  Luc Van Gool,et al.  GPU-Based Foreground-Background Segmentation using an Extended Colinearity Criterion , 2005 .

[14]  Matt Pharr,et al.  Gpu gems 2: programming techniques for high-performance graphics and general-purpose computation , 2005 .

[15]  Minglun Gong,et al.  Real-Time Image Processing Using Graphics Hardware: A Performance Study , 2005, ICIAR.

[16]  Steve Mann,et al.  OpenVIDIA: parallel GPU computer vision , 2005, ACM Multimedia.

[17]  Kevin Bjorke.  Image processing on parallel GPU pixel units , 2006, Electronic Imaging.

[18]  Jean-Philippe Pons,et al.  A GPU Implementation of Level Set Multiview Stereo , 2006, International Conference on Computational Science.

[19]  Jan-Michael Frahm,et al.  Towards Urban 3D Reconstruction from Video , 2006, Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06).

[20]  Stan Birchfield.  Derivation of Kanade-Lucas-Tomasi Tracking Equation , 2006 .