Real-time Hand Tracking With Variable-Length Markov Models of Behaviour

We present a novel approach for visual tracking of structured behaviour as observed in human-computer interaction. An automatically acquired variable-length Markov model is used to represent the high-level structure and temporal ordering of gestures. Continuous estimation of hand posture is handled by combining the model with annealed particle filtering. The stochastic simulation updates, and automatically switches between, different model representations of hand posture that correspond to distinct gestures. The implementation executes in real time and demonstrates significant improvement in robustness over comparable methods. We provide a measurement of user performance when our method is applied to a Fitts’ law drag-anddrop task, and an analysis of the effects of latency that it introduces.

[1]  David C. Hogg,et al.  Wormholes in shape space: tracking through discontinuous changes in shape , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[2]  Michael Isard,et al.  A mixed-state condensation tracker with automatic model-switching , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[3]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[4]  Björn Stenger,et al.  Learning a Kinematic Prior for Tree-Based Filtering , 2003, BMVC.

[5]  Robert S. Allison,et al.  Tolerance of temporal delay in virtual environments , 2001, Proceedings IEEE Virtual Reality 2001.

[6]  N. Stefanov,et al.  A robust, self-initialising, real-time hand tracker , 2005 .

[7]  Yoichi Sato,et al.  Real-Time Fingertip Tracking and Gesture Recognition , 2002, IEEE Computer Graphics and Applications.

[8]  Michael Isard,et al.  Contour Tracking by Stochastic Propagation of Conditional Density , 1996, ECCV.

[9]  Yoram Singer,et al.  Batch and On-Line Parameter Estimation of Gaussian Mixtures Based on the Joint Entropy , 1998, NIPS.

[10]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[11]  David C. Hogg,et al.  Learning Variable-Length Markov Models of Behavior , 2001, Comput. Vis. Image Underst..

[12]  M. Young,et al.  Discrimination of Changes of Latency during Voluntary Hand Movement of Virtual Objects , 1999 .

[13]  Neil J. Gordon,et al.  A tutorial on particle filters for online nonlinear/non-Gaussian Bayesian tracking , 2002, IEEE Trans. Signal Process..

[14]  Hideo Saito,et al.  Virtual 3-D interface system via hand motion recognition from two cameras , 2002, IEEE Trans. Syst. Man Cybern. Part A.

[15]  Anthony G. Cohn,et al.  Modeling Interaction Using Learnt Qualitative Spatio-Temporal Relations and Variable Length Markov Models , 2002, ECAI.

[16]  Stuart K. Card,et al.  Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys, for text selection on a CRT , 1987 .

[17]  Tieniu Tan,et al.  Real time hand tracking by combining particle filtering and mean shift , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[18]  Dana Ron,et al.  The Power of Amnesia , 1993, NIPS.

[19]  Greg Welch,et al.  Welch & Bishop , An Introduction to the Kalman Filter 2 1 The Discrete Kalman Filter In 1960 , 1994 .

[20]  Rangachar Kasturi,et al.  Machine vision , 1995 .

[21]  I. MacKenzie,et al.  A note on the information-theoretic basis of Fitts' law. , 1989, Journal of motor behavior.

[22]  MacKenzie Is.  A Note on the Information-Theoretic Basis for Fitts’ Law , 1989 .

[23]  Yasushi Yagi,et al.  A System for 3D Motion and Position Estimation of Hand from Monocular Image Sequence , 1995 .

[24]  Isabelle Guyon,et al.  Design of a linguistic postprocessor using variable memory length Markov models , 1995, Proceedings of 3rd International Conference on Document Analysis and Recognition.

[25]  G. Kitagawa.  Smoothness priors analysis of time series , 1996 .

[26]  N. Gordon,et al.  Novel approach to nonlinear/non-Gaussian Bayesian state estimation , 1993 .