Fusion of Human Posture Features for Continuous Action Recognition

This paper presents a real-time or online system for continuous recognition of human actions. The system recognizes actions such as walking, bending, jumping, waving, and falling and relies on spatial features computed to characterize human posture. The paper evaluates the utility of these features based on its joint or independent treatment within the context of the Hidden Markov Model (HMM) framework. A baseline approach wherein disparate spatial features are treated as an input vector to trained HMMs is used to compare three different independent feature models. In addition, an action transition constraints is introduced to stabilize the developed models and allow for continuity in recognized actions. The system is evaluated across a dataset of videos and results reported in terms of frame error rate, the frame delay in recognizing an action, action recognition rate, and the missed and false recognition rates. Experimental results shows the effectiveness of the proposed treatment of input features and the corresponding HMM formulations.

[1]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[2]  Xinghua Sun,et al.  Action recognition via local descriptors and holistic features , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[3]  Josef Kittler,et al.  Combining classifiers: A theoretical framework , 1998, Pattern Analysis and Applications.

[4]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[5]  S. Miaou,et al.  A Customized Human Fall Detection System Using Omni-Camera Images and Personal Information , 2006, 1st Transdisciplinary Conference on Distributed Diagnosis and Home Healthcare, 2006. D2H2..

[6]  Ivan Laptev,et al.  On Space-Time Interest Points , 2005, International Journal of Computer Vision.

[7]  Rama Chellappa,et al.  Video Metrology Using a Single Camera , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Matti Pietikäinen,et al.  Human Activity Recognition Using Sequences of Postures , 2005, MVA.

[9]  Eli Shechtman,et al.  Space-time behavior based correlation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[10]  Larry S. Davis,et al.  Model-based object pose in 25 lines of code , 1992, International Journal of Computer Vision.

[11]  Masahiko Yachida,et al.  Real-time context-based gesture recognition using HMM and automaton , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[12]  Serge J. Belongie,et al.  Behavior recognition via sparse spatio-temporal features , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[13]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[14]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Juan Carlos Niebles,et al.  A Hierarchical Model of Shape and Appearance for Human Action Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Jake K. Aggarwal,et al.  Segmentation and recognition of continuous human activity , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[17]  Ronen Basri,et al.  Actions as Space-Time Shapes , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[19]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[20]  N. Otsu,et al.  Action and simultaneous multiple-person identification using cubic higher-order local auto-correlation , 2004, ICPR 2004.

[21]  Gerhard Rigoll,et al.  Hidden Markov model based continuous online gesture recognition , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[22]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[23]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Zhihai He,et al.  Recognizing Falls from Silhouettes , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[25]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[26]  Pedro Ribeiro,et al.  Human Activity Recognition from Video: modeling, feature selection and classification architecture , 2005 .

[27]  Juan Carlos Niebles,et al.  Unsupervised Learning of Human Action Categories , 2006 .