Real-time motion estimation and visualization on graphics cards

We present a tool for real-time visualization of motion features in 2D image sequences. The motion is estimated through an eigenvector analysis of the spatio-temporal structure tensor at every pixel location. This approach is computationally demanding but allows reliable velocity estimates as well as quality indicators for the obtained results. We use a 2D color map and a region of interest selector for the visualization of the velocities. On the selected velocities we apply a hierarchical smoothing scheme which allows the choice of the desired scale of the motion field. We demonstrate several examples of test sequences in which some persons are moving with different velocities than others. These persons are visually marked in the real-time display of the image sequence. The tool is also applied to angiography sequences to emphasize the blood flow and its distribution. An efficient processing of the data streams is achieved by mapping the operations onto the stream architecture of standard graphics cards. The card receives the images and performs both the motion estimation and visualization, taking advantage of the parallelism in the graphics processor and the superior memory bandwidth. The integration of data processing and visualization also saves on unnecessary data transfers and thus allows the real-time analysis of 320/spl times/240 images. We expect that on the newest generation of graphics hardware our tool could run in real time for the standard VGA format.

[1]  Takeo Kanade,et al.  Introduction to the Special Section on Video Surveillance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[2]  Signal Based Surveillance.  Proceedings IEEE Conference on Advanced Video and Signal Based Surveillance. AVSS 2003 , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[3]  Rüdiger Westermann,et al.  Linear algebra operators for GPU implementation of numerical algorithms , 2003, SIGGRAPH Courses.

[4]  David J. Fleet,et al.  Computing Optical Flow with Physical Models of Brightness Variation , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  H. Knutsson.  Representing Local Structure Using Tensors , 1989 .

[6]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[7]  Jens H. Krüger,et al.  GPGPU: general purpose computation on graphics hardware , 2004, SIGGRAPH '04.

[8]  Robert Strzodka,et al.  Generalized distance transforms and skeletons in graphics hardware , 2004, VISSYM'04.

[9]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[10]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[11]  Ivan Viola,et al.  Fast and Flexible High-Quality Texture Filtering With Tiled High-Resolution Filters , 2002, VMV.

[12]  Johan Wiklund,et al.  Multidimensional Orientation Estimation with Applications to Texture Analysis and Optical Flow , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Thomas Ertl,et al.  Accelerating 3D convolution using graphics hardware (case study) , 1999 .

[14]  Min Chen,et al.  Video visualization , 2003 .

[15]  Jarke J. van Wijk,et al.  Image based flow visualization , 2002, ACM Trans. Graph..

[16]  Jin Wu,et al.  Mean square slopes of the wind-disturbed water surface, their magnitude, directionality, and composition , 1990 .

[17]  Shyang Chang,et al.  Statistical change detection with moments under time-varying illumination , 1998, IEEE Trans. Image Process..

[18]  Zlatko Drmac,et al.  Implementation of Jacobi Rotations for Accurate Singular Value Computation in Floating Point Arithmetic , 1997, SIAM J. Sci. Comput..

[19]  Carl-Fredrik Westin,et al.  Level Set Based Integration of Segmentation and Computational Fluid Dynamics for Flow Correction in Phase Contrast Angiography , 2002, MICCAI.

[20]  Erhardt Barth,et al.  Analytic solutions for multiple motions , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[21]  Anselmo Lastra,et al.  Radiosity on graphics hardware , 2004, Graphics Interface.

[22]  Hagen Spies,et al.  Motion , 2000, Computer Vision and Applications.

[23]  José Luis Martín,et al.  Hardware architecture for optical flow estimation in real time , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[24]  Greg Humphreys,et al.  A multigrid solver for boundary value problems using programmable graphics hardware , 2003, HWWS '03.

[25]  Francis K. H. Quek,et al.  A review of vessel extraction techniques and algorithms , 2004, CSUR.

[26]  Ted Camus.  Real-Time Quantized Optical Flow , 1997, Real Time Imaging.

[27]  Alessandro Verri,et al.  Computing optical flow from an overconstrained system of linear algebraic equations , 1990, [1990] Proceedings Third International Conference on Computer Vision.

[28]  Hanno Scharr,et al.  Principles of Filter Design , 1999 .

[29]  Timo Kohlberger,et al.  Domain Decomposition for Parallel Variational Optical Flow Computation , 2003, DAGM-Symposium.

[30]  Philippe Colantoni,et al.  Fast and Accurate Color Images Processing Using 3D Graphics Cards , 2003, VMV.

[31]  David J. Fleet,et al.  Computing optical flow with physical models of brightness variation , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[32]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[33]  Claude L. Fennema,et al.  Velocity determination in scenes containing several moving objects , 1979 .

[34]  Thomas Ertl,et al.  Accelerating 3D convolution using graphics hardware , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[35]  Miguel O. Arias-Estrada,et al.  FPGA Processor for Real-Time Optical Flow Computation , 2003, FPL.

[36]  Gordon Erlebacher,et al.  A texture-based framework for spacetime-coherent visualization of time-dependent vector fields , 2003, IEEE Visualization, 2003. VIS 2003..

[37]  Min Chen,et al.  Video visualization , 2003, IEEE Visualization, 2003. VIS 2003..

[38]  Ross T. Whitaker,et al.  Interactive deformation and visualization of level set surfaces using graphics hardware , 2003, IEEE Visualization, 2003. VIS 2003..

[39]  Ellen C. Hildreth,et al.  Computations Underlying the Measurement of Visual Motion , 1984, Artif. Intell..

[40]  Eitan Grinspun,et al.  Sparse matrix solvers on the GPU: conjugate gradients and multigrid , 2003, SIGGRAPH Courses.

[41]  Ethan K. Brodsky,et al.  Interactive Visualization of Time-Resolved Contrast-Enhanced Magnetic Resonance Angiography (CE-MRA) , 2003 .

[42]  William H. Press,et al.  Numerical recipes in C , 2002 .