Real-time human action classification using a dynamic neural model

The multiple timescale recurrent neural network (MTRNN) model is a useful tool for recording and regenerating a continuous signal for dynamic tasks. However, our research shows that the MTRNN model is difficult to use for the classification of multiple types of motion when observing a human action. Therefore, in this paper, we propose a new supervised MTRNN model for handling the issue of action classification. Instead of setting the initial states, we define a group of slow context nodes as "classification nodes." The supervised MTRNN model provides both prediction and classification outputs simultaneously during testing. Our experiment results show that the supervised MTRNN model inherits the basic function of an MTRNN and can be used to generate action signals. In addition, the results show that the robustness of the supervised MTRNN model is better than that of the MTRNN model when generating both action sequences and action classification tasks.

[1]  C. Keysers,et al.  The Observation and Execution of Actions Share Motor and Somatosensory Voxels in all Tested Subjects: Single-Subject Analyses of Unsmoothed fMRI Data , 2008, Cerebral cortex.

[2]  Minho Lee,et al.  Neuro-robotics study on integrative learning of proactive visual attention and motor behaviors , 2012, Cognitive Neurodynamics.

[3]  Shigeki Sugano,et al.  Imitating others by composition of primitive actions: A neuro-dynamic model , 2012, Robotics Auton. Syst..

[4]  A. Leslie.  Pretense and representation: The origins of "theory of mind." , 1987 .

[5]  Jun Tani,et al.  Emergence of Functional Hierarchy in a Multiple Timescale Neural Network Model: A Humanoid Robot Experiment , 2008, PLoS Comput. Biol..

[6]  Minho Lee,et al.  Supervised Multiple Timescale Recurrent Neuron Network Model for Human Action Classification , 2013, ICONIP.

[7]  Shigeki Sugano,et al.  CREATING NOVEL GOAL-DIRECTED ACTIONS AT CRITICALITY: A NEURO-ROBOTIC EXPERIMENT , 2009 .

[8]  N. Altman.  An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[9]  Reza Shadmehr,et al.  Learning of action through adaptive combination of motor primitives , 2000, Nature.

[10]  Tetsuya Ogata,et al.  Emergence of hierarchical structure mirroring linguistic composition in a recurrent neural network , 2011, Neural Networks.

[11]  Luiz Velho,et al.  Kinect and RGBD Images: Challenges and Applications , 2012, 2012 25th SIBGRAPI Conference on Graphics, Patterns and Images Tutorials.

[12]  Tanja Schultz,et al.  HMM-based human motion recognition with optical flow data , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[13]  A. Goldman.  Imitation, Mind Reading, and Simulation. , 2005 .

[14]  Kenji Doya,et al.  Adaptive neural oscillator using continuous-time back-propagation learning , 1989, Neural Networks.

[15]  Yang Zhang,et al.  Classification of Known and Unknown Environmental Sounds Based on Self-Organized Space Using a Recurrent Neural Network , 2011, Adv. Robotics.

[16]  G. Rizzolatti,et al.  Parietal Lobe: From Action Organization to Intention Understanding , 2005, Science.

[17]  D. Wolpert,et al.  Mental state inference using visual control parameters. , 2005, Brain research. Cognitive brain research.

[18]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[19]  Yuichi Nakamura,et al.  Approximation of dynamical systems by continuous time recurrent neural networks , 1993, Neural Networks.

[20]  Peter Stagge,et al.  Recurrent neural networks for time series classification , 2003, Neurocomputing.

[21]  H. H. Avilés-Arriaga,et al.  A Comparison of Dynamic Naive Bayesian Classifiers and Hidden , 2011 .

[22]  G. Rizzolatti,et al.  Premotor cortex and the recognition of motor actions. , 1996, Brain research. Cognitive brain research.

[23]  Jun Tani,et al.  Self-organization of distributedly represented multiple behavior schemata in a mirror system: reviews of robot experiments using RNNPB , 2004, Neural Networks.

[24]  Qing Chen,et al.  Dynamic Gesture Recognition , 2005, 2005 IEEE Instrumentationand Measurement Technology Conference Proceedings.

[25]  Jun Tani,et al.  Achieving "organic compositionality" through self-organization: Reviews on brain-inspired robotics experiments , 2008, Neural Networks.