Recognition of Human Actions using an Optimal Control Based Motor Model

We present a novel approach to the problem of representation and recognition of human actions, that uses an optimal control based model to connect the high-level goals of a human subject to the low-level movement trajectories captured by a computer vision system. These models quantify the high-level goals as a performance criterion or cost function which the human sensorimotor system optimizes by picking the control strategy that achieves the best possible performance. We show that the human body can be modeled as a hybrid linear system that can operate in one of several possible modes, where each mode corresponds to a particular high-level goal or cost function. The problem of action recognition, then is to infer the current mode of the system from observations of the movement trajectory. We demonstrate our approach on 3D visual data of human arm motion.

[1]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Daniel M. Wolpert,et al.  Signal-dependent noise determines motor planning , 1998, Nature.

[3]  Ramakant Nevatia,et al.  3D Human Action Recognition Using Spatio-temporal Motion Templates , 2005, ICCV-HCI.

[4]  Pietro Perona,et al.  Decomposition of human motion into dynamics-based primitives with application to drawing tasks , 2003, Autom..

[5]  Jessica K. Hodgins,et al.  Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces , 2004, SIGGRAPH 2004.

[6]  Jernej Barbic,et al.  Segmenting Motion Capture Data into Distinct Behaviors , 2004, Graphics Interface.

[7]  James M. Rehg,et al.  Learning and inference in parametric switching linear dynamic systems , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[8]  Jake K. Aggarwal,et al.  A hierarchical Bayesian network for event recognition of human actions and interactions , 2004, Multimedia Systems.

[9]  Brian C. Williams,et al.  Mode Estimation of Probabilistic Hybrid Systems , 2002, HSCC.

[10]  E. Todorov.  Optimality principles in sensorimotor control , 2004, Nature Neuroscience.

[11]  A.D. Kuo,et al.  An optimal control model for analyzing human postural balance , 1995, IEEE Transactions on Biomedical Engineering.

[12]  Y. Boers,et al.  Interacting multiple model particle filter , 2003 .

[13]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficients , 1988 .

[14]  Jessica K. Hodgins,et al.  Synthesizing physically realistic human motion in low-dimensional, behavior-specific spaces , 2004, ACM Trans. Graph..

[15]  Ruggero Frezza,et al.  Control of a Manipulator with a Minimum Number of Motion Primitives , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[16]  Rémi Ronfard,et al.  Automatic Discovery of Action Taxonomies from Multiple Views , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  George W. Irwin,et al.  Multiple model bootstrap filter for maneuvering target tracking , 2000, IEEE Trans. Aerosp. Electron. Syst..

[18]  Nando de Freitas,et al.  Sequential Monte Carlo Methods in Practice , 2001, Statistics for Engineering and Information Science.

[19]  James M. Rehg,et al.  Learning and Inferring Motion Patterns using Parametric Segmental Switching Linear Dynamic Systems , 2008, International Journal of Computer Vision.

[20]  Emanuel Todorov,et al.  Iterative Linear Quadratic Regulator Design for Nonlinear Biological Movement Systems , 2004, ICINCO.

[21]  Alex Pentland,et al.  Coupled hidden Markov models for complex action recognition , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.

[23]  Eric Horvitz,et al.  Layered representations for learning and inferring office activity from multiple sensory channels , 2004, Comput. Vis. Image Underst..

[24]  Maja J. Mataric,et al.  Automated Derivation of Primitives for Movement Classification , 2000, Auton. Robots.

[25]  S. Scott.  Optimal feedback control and the neural basis of volitional motor control , 2004, Nature Reviews Neuroscience.

[26]  Y. Bar-Shalom,et al.  The interacting multiple model algorithm for systems with Markovian switching coefficientsY. Bar-Shalom , 1988 .

[27]  N. D. Freitas.  Rao-Blackwellised particle filtering for fault diagnosis , 2002 .

[28]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[29]  Ruzena Bajcsy,et al.  Skeleton-Based Data Compression for Multi-camera Tele-Immersion System , 2007, ISVC.

[30]  Michael K. Pitt,et al.  Auxiliary Variable Based Particle Filters , 2001, Sequential Monte Carlo Methods in Practice.