Modeling and Prediction of Human Behavior

We propose that many human behaviors can be accurately described as a set of dynamic models (e.g., Kalman filters) sequenced together by a Markov chain. We then use these dynamic Markov models to recognize human behaviors from sensory data and to predict human behaviors over a few seconds time. To test the power of this modeling approach, we report an experiment in which we were able to achieve 95 accuracy at predicting automobile drivers' subsequent actions from their initial preparatory movements.

[1]  Michael I. Jordan,et al.  Learning Fine Motion by Markov Mixtures of Experts , 1995, NIPS.

[2]  Alan S. Willsky.  Detection of abrupt changes in dynamic systems , 1985 .

[3]  Yangsheng Xu,et al.  Human action learning via hidden Markov model , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[4]  R. E. Kalman,et al.  New Results in Linear Filtering and Prediction Theory , 1961 .

[5]  Alex Pentland,et al.  Synchronization in Virtual Realities , 1992, Presence: Teleoperators & Virtual Environments.

[6]  Richard M. Schwartz,et al.  On-line cursive handwriting recognition using speech recognition methods , 1994, Proceedings of ICASSP '94. IEEE International Conference on Acoustics, Speech and Signal Processing.

[7]  Alex Pentland,et al.  Device synchronization using an optimal linear filter , 1992, I3D '92.

[8]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[9]  L. Baum,et al.  An inequality and associated maximization technique in statistical estimation of probabilistic functions of a Markov process , 1972 .

[10]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  A. Pentland,et al.  Toward augmented control systems , 1995, Proceedings of the Intelligent Vehicles '95. Symposium.

[12]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[13]  Rajesh P. N. Rao,et al.  Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex , 1997, Neural Computation.

[14]  Michèle Basseville,et al.  Detection of Abrupt Changes in Signals and Dynamical Systems , 1985 .

[15]  Christoph Bregler,et al.  Learning and recognizing human dynamics in video sequences , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Ramesh C. Jain,et al.  Dynamic vision , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.

[17]  J. P. Hartnett,et al.  The Influence of Prandtl Number on the Heat Transfer From Rotating Nonisothermal Disks and Cones , 1961 .

[18]  A. Pentland,et al.  Method for evaluating human and simulated drivers in real traffic situations , 1996, Proceedings of Vehicular Technology Conference - VTC.

[19]  B.-H. Juang,et al.  Maximum-likelihood estimation for mixture multivariate stochastic observations of Markov chains , 1985, AT&T Technical Journal.

[20]  Alex Pentland,et al.  Looking at People: Sensing for Ubiquitous and Wearable Computing , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .