Human Action Recognition Using Time Delay Input Radial Basis Function Networks

This paper presents a fast, vision-based method for the problem of human action representation and recognition. The first problem is addressed by constructing an action descriptor from spatiotemporal data of action silhouettes based on appearance and motion features. For action classification, a new Radial Basis Function Network (RBF), called Time Delay Input Radial Basis Function Network (TDIRBF) is proposed by introducing time delay units to the RBF in a novel approach. A TDIRBF offers a few desirable features such as an easier learning process and more flexibility. The representational power and speed of the proposed method were explored using a publicly available dataset. Based on experimental results, implemented in MATLAB and on standard PCs, the average time for constructing a feature vector for a high-resolution video was just about 20 ms/frame (or 50 fps) and the classifier speed was above 15 fps. Furthermore, the proposed approach demonstrated good performance in terms of both execution time and overall performance (a new performance measure that combines accuracy and speed into one metric).

[1]  Rama Chellappa,et al.  Accuracy vs Efficiency Trade-offs in Optical Flow Algorithms , 1996, Comput. Vis. Image Underst..

[2]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Junxia Gu,et al.  Action and Gait Recognition From Recovered 3-D Human Joints , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Tanaya Guha,et al.  Learning Sparse Representations for Human Action Recognition , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Andrew Hunter,et al.  Application of the self-organising map to trajectory classification , 2000, Proceedings Third IEEE International Workshop on Visual Surveillance.

[6]  Hassan Foroosh,et al.  View-Invariant Action Recognition from Point Triplets , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Chia-Feng Juang,et al.  Moving object recognition by a shape-based neural fuzzy network , 2008, Neurocomputing.

[8]  Seong-Whan Lee,et al.  View-independent human action recognition with Volume Motion Template on single stereo camera , 2010, Pattern Recognit. Lett..

[9]  Ze-Nian Li,et al.  Action Detection in Cluttered Video With Successive Convex Matching , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  A F Bobick,et al.  Movement, activity and action: the role of knowledge in the perception of motion. , 1997, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[11]  Jesse Hoey,et al.  Representation and recognition of complex human motion , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[12]  Larry S. Davis,et al.  Recognizing actions by shape-motion prototype trees , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[13]  Du Tran,et al.  Human Activity Recognition with Metric Learning , 2008, ECCV.

[14]  Rémi Ronfard,et al.  A survey of vision-based methods for action representation, segmentation and recognition , 2011, Comput. Vis. Image Underst..

[15]  Ling Shao,et al.  Embedding Motion and Structure Features for Action Recognition , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[17]  Alex Pentland,et al.  Human computing and machine understanding of human behavior: a survey , 2006, ICMI '06.

[18]  Peyman Milanfar,et al.  Action Recognition from One Example , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[20]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[21]  Rama Chellappa,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Matching Shape Sequences in Video with Applications in Human Movement Analysis. Ieee Transactions on Pattern Analysis and Machine Intelligence 2 , 2022 .

[22]  Rémi Ronfard,et al.  Free viewpoint action recognition using motion history volumes , 2006, Comput. Vis. Image Underst..

[23]  Tanveer F. Syeda-Mahmood,et al.  Invariance in motion analysis of videos , 2003, ACM Multimedia.

[24]  Hilary Buxton,et al.  Learning identity with radial basis function networks , 1998, Neurocomputing.

[25]  Pau-Choo Chung,et al.  A daily behavior enabled hidden Markov model for human behavior understanding , 2008, Pattern Recognit..

[26]  Mohamad T. Musavi,et al.  On the training of radial basis function classifiers , 1992, Neural Networks.

[27]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[28]  Tiziana D'Orazio,et al.  A review of vision-based systems for soccer video analysis , 2010, Pattern Recognit..

[29]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[30]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[31]  Sudeep Sarkar,et al.  Distribution-Based Dimensionality Reduction Applied to Articulated Motion Recognition , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Alexandros Iosifidis,et al.  View-Invariant Action Recognition Based on Artificial Neural Networks , 2012, IEEE Transactions on Neural Networks and Learning Systems.

[33]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[34]  Ronen Basri,et al.  Actions as Space-Time Shapes , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Eduard Petlenkov,et al.  Application of self organizing Kohonen map to detection of surgeon motions during endoscopic surgery , 2008, 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).

[37]  Weidong Geng,et al.  Example-Based Automatic Music-Driven Conventional Dance Motion Synthesis , 2012, IEEE Transactions on Visualization and Computer Graphics.

[38]  Yuntao Cui,et al.  Learning-based hand sign recognition using SHOSLIF-M , 1995, Proceedings of IEEE International Conference on Computer Vision.

[39]  Yang Wang,et al.  Human Action Recognition by Semilatent Topic Models , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[41]  Q. M. Jonathan Wu,et al.  Incremental Learning in Human Action Recognition Based on Snippets , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Ling Shao,et al.  Silhouette Analysis-Based Action Recognition Via Exploiting Human Poses , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Woontack Woo,et al.  Emotion Recognition from Dance Image Sequences Using Contour Approximation , 2004, SSPR/SPR.

[44]  Mark S. Nixon,et al.  Heel strike detection based on human walking movement for surveillance analysis , 2013, Pattern Recognit. Lett..

[45]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[46]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[47]  Mubarak Shah,et al.  Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[49]  R. Venkatesh Babu,et al.  Recognition of human actions using motion history information extracted from the compressed video , 2004, Image Vis. Comput..

[50]  Wei Huang,et al.  Human action recognition based on Self Organizing Map , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[51]  Michael R. Berthold,et al.  A time delay radial basis function network for phoneme recognition , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[52]  Michael J. Black,et al.  Parameterized Modeling and Recognition of Activities , 1999, Comput. Vis. Image Underst..

[53]  James W. Davis,et al.  An appearance-based representation of action , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[54]  Narendra Ahuja,et al.  Extraction of 2D Motion Trajectories and Its Application to Hand Gesture Recognition , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[55]  Azriel Rosenfeld,et al.  Sequential Operations in Digital Picture Processing , 1966, JACM.