Tracking and Clustering Salient Features in Image Sequences

Many applications in media production need information about moving objects in the scene, e.g. insertion of computer-generated objects, association of sound sources to these objects or visualization of object trajectories in broadcasting. We present a GPU accelerated approach for detecting and tracking salient features in image sequences and we propose an algorithm for clustering the obtained feature point trajectories in order to obtain a motion segmentation of the set of feature trajectories. Evaluation results for both the tracking and clustering steps are presented. Finally we discuss the application of the proposed approach for associating audio sources to objects to support audio rendering for virtual sets.

[1]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[2]  Samuel Pachoud,et al.  Grouping motion trajectories , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[3]  Yakup Genc,et al.  GPU-based Video Feature Tracking And Matching , 2006 .

[4]  Padhraic Smyth,et al.  Clustering Sequences with Hidden Markov Models , 1996, NIPS.

[5]  René Vidal,et al.  A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Pau Arumí,et al.  Real‐time 3D audio for digital cinema , 2008 .

[7]  Patrick Pérez,et al.  Clustering Point Trajectories with Various Life-Spans , 2009, 2009 Conference for Visual Media Production.

[8]  Takeo Kanade,et al.  A multi-body factorization method for motion analysis , 1995, Proceedings of IEEE International Conference on Computer Vision.

[9]  Hannes Fassold,et al.  Realtime KLT Feature Point Tracking for High Definition Video , 2009 .

[10]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[11]  George Kollios,et al.  Extraction and clustering of motion trajectories in video , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[12]  David G. Stork,et al.  Pattern Classification , 1973 .

[13]  Carl Tim Kelley,et al.  Iterative methods for optimization , 1999, Frontiers in applied mathematics.

[14]  Takeo Kanade,et al.  Shape and motion from image streams under orthography: a factorization method , 1992, International Journal of Computer Vision.

[15]  Kenichi Kanatani,et al.  Motion segmentation by subspace separation and model selection , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Jan-Michael Frahm,et al.  Fast gain-adaptive KLT tracking on the GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[17]  Nicholas J. Redding,et al.  GPU-Accelerated KLT Tracking with Monte-Carlo-Based Feature Reselection , 2008, 2008 Digital Image Computing: Techniques and Applications.

[18]  Jaume Durany,et al.  ACOUSTICS2008/2846 Towards a realistic ray tracing for room acoustics , 2008 .

[19]  Koichiro Hiyama,et al.  5.1 and 22.2 Multichannel Sound Productions Using an Integrated Surround Sound Panning System , 2004 .