Extraction and temporal segmentation of multiple motion trajectories in human motion

A new method for extraction and temporal segmentation of multiple motion trajectories in human motion is presented. The proposed method extracts motion trajectories generated by body parts without any initialization or any assumption on color distribution. Motion trajectories are very compact and representative features for activity recognition. Tracking human body parts (hands and feet) is inherently difficult because the body parts which generate most of the motion trajectories are relatively small compared to the human body. This problem is overcome by using a new motion segmentation method: at every frame, candidate motion locations are detected and set as significant motion points (SMPs). The motion trajectories are obtained by combining these SMPs and the color-optical flow based tracker results. These motion trajectories are inturn used as features for temporal segmentation of specific activities from continuous video sequences. The proposed approach is tested on actual ballet step sequences. Experimental results show that the proposed method can successfully extract and temporally segment multiple motion trajectories from human motion.

[1]  M.L. Miller,et al.  A comparison of two algorithms for determining ranked assignments with application to multitarget tracking and motion correspondence , 1997, IEEE Transactions on Aerospace and Electronic Systems.

[2]  Jin Hyeong Park,et al.  Extraction of Multiple Motion Trajectories in Human Motion , 2003, SCIA.

[3]  Sudeep Sarkar,et al.  Studies on silhouette quality and gait recognition , 2004, CVPR 2004.

[4]  Dorin Comaniciu,et al.  Kernel-Based Object Tracking , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Gérard G. Medioni,et al.  Detecting and tracking moving objects for video surveillance , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[6]  Y. Weiss,et al.  Multibody factorization with uncertainty and missing data using the EM algorithm , 2004, CVPR 2004.

[7]  M. Nixon,et al.  Extraction and recognition of periodically deforming objects by continuous, spatio-temporal shape description , 2004, CVPR 2004.

[8]  Aaron F. Bobick,et al.  Learning visual behavior for gesture analysis , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[9]  Michael J. Black,et al.  A Probabilistic Framework for Matching Temporal Trajectories: CONDENSATION-Based Recognition of Gestures and Expressions , 1998, ECCV.

[10]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[11]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[12]  Gregory D. Hager,et al.  Probabilistic Data Association Methods for Tracking Complex Visual Objects , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  R. Vidal,et al.  Motion segmentation with missing data using PowerFactorization and GPCA , 2004, CVPR 2004.

[14]  Patrick Bouthemy,et al.  Probabilistic parameter-free motion detection , 2004, CVPR 2004.

[15]  David Suter,et al.  Robust Optic Flow Computation , 1998, International Journal of Computer Vision.

[16]  Ming Ye,et al.  Estimating Piecewise-Smooth Optical Flow with Global Matching and Graduated Optimization , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Mubarak Shah,et al.  View-Invariant Representation and Recognition of Actions , 2002, International Journal of Computer Vision.

[18]  Rangachar Kasturi,et al.  Extraction and Temporal Segmentation of Multiple Motion Trajectories in Human Motion , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[19]  Yizong Cheng,et al.  Mean Shift, Mode Seeking, and Clustering , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Rangachar Kasturi,et al.  Activity recognition based on multiple motion trajectories , 2004, ICPR 2004.

[21]  Janusz Konrad,et al.  Multiple motion segmentation with level sets , 2003, IEEE Trans. Image Process..

[22]  Mubarak Shah,et al.  Establishing motion correspondence , 1991, CVGIP Image Underst..

[23]  Aaron F. Bobick,et al.  Parametric Hidden Markov Models for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Mei Han,et al.  An algorithm for multiple object trajectory tracking , 2004, CVPR 2004.

[25]  Jin-Hyung Kim,et al.  An HMM-Based Threshold Model Approach for Gesture Recognition , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Cor J. Veenman,et al.  Resolving Motion Correspondence for Densely Moving Points , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  R. Collins,et al.  Representation and matching of articulated shapes , 2004, CVPR 2004.

[28]  Aaron E. Rosenberg,et al.  Performance tradeoffs in dynamic time warping algorithms for isolated word recognition , 1980 .

[29]  Aaron F. Bobick,et al.  A State-Based Approach to the Representation and Recognition of Gesture , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Dorin Comaniciu,et al.  Mean Shift: A Robust Approach Toward Feature Space Analysis , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Tanveer F. Syeda-Mahmood,et al.  View-invariant alignment and matching of video sequences , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[32]  P. Anandan,et al.  A computational framework and an algorithm for the measurement of visual motion , 1987, International Journal of Computer Vision.

[33]  Pietro Perona,et al.  Scene segmentation from 3D motion , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[34]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[35]  Rachid Deriche,et al.  Geodesic Active Contours and Level Sets for the Detection and Tracking of Moving Objects , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Bir Bhanu,et al.  Statistical feature fusion for gait-based human recognition , 2004, CVPR 2004.

[37]  Donald Reid.  An algorithm for tracking multiple targets , 1978 .

[38]  Ingemar J. Cox,et al.  An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Ying Wu,et al.  Multibody motion segmentation based on simulated annealing , 2004, CVPR 2004.

[40]  David Burshtein,et al.  An enhanced dynamic time warping model for improved estimation of DTW parameters , 2003, IEEE Trans. Speech Audio Process..

[41]  Michael J. Black,et al.  The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields , 1996, Comput. Vis. Image Underst..

[42]  Narendra Ahuja,et al.  Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[43]  Mubarak Shah,et al.  Motion trajectories , 1993, IEEE Trans. Syst. Man Cybern..

[44]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.