Learning visual behavior for gesture analysis

A state-based method for learning visual behavior from image sequences is presented. The technique is novel for its incorporation of multiple representations into the Hidden Markov Model framework. Independent representations of the instantaneous visual input at each state of the Markov model are estimated concurrently with the learning of the temporal characteristics. Measures of the degree to which each representation describes the input are combined to determine an input's overall membership to a state. We exploit two constraints allowing application of the technique to view-based gesture recognition: gestures are modal in the space of possible human motion, and gestures are viewpoint-dependent. The recovery of the visual behavior of a number of simple gestures with a small number of low resolution image sequences is shown.

[1]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[2]  Hiroshi Murase,et al.  Learning and recognition of 3D objects from appearance , 1993, [1993] Proceedings IEEE Workshop on Qualitative Vision.

[3]  J. O'Rourke,et al.  Model-based image analysis of human motion using constraint propagation , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Edward Hunter,et al.  Vision based hand gesture interpretation using recursive estimation , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[5]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[6]  Ramesh C. Jain,et al.  Recursive identification of gesture inputs using hidden Markov models , 1994, Proceedings of 1994 IEEE Workshop on Applications of Computer Vision.

[7]  Shuji Hashimoto,et al.  A computer music system that follows a human conductor , 1991, Computer.

[8]  Michael S. Landy,et al.  Intelligible encoding of ASL image sequences at extremely low information rates , 1985, Comput. Vis. Graph. Image Process..

[9]  G Sperling,et al.  Intelligent temporal subsampling of American Sign Language using event boundaries. , 1990, Journal of experimental psychology. Human perception and performance.

[10]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[11]  Alex Pentland,et al.  View-based and modular eigenspaces for face recognition , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[12]  L. Rabiner,et al.  An introduction to hidden Markov models , 1986, IEEE ASSP Magazine.

[13]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[14]  R. F. Rashid,et al.  Towards a system for the interpretation of moving light displays , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Aaron F. Bobick,et al.  Recognition of human body motion using phase space constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[16]  E. Catmull,et al.  A CLASS OF LOCAL INTERPOLATING SPLINES , 1974 .

[17]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[18]  J. Sklansky,et al.  Segmentation of people in motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[19]  Mubarak Shah,et al.  A survey of motion analysis from moving light displays , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[20]  S. Edelman.  Representation of Similarity in 3D Object Discrimination , 1995 .

[21]  Mubarak Shah,et al.  The trajectory primal sketch: a multi-scale scheme for representing motion characteristics , 1989, Proceedings CVPR '89: IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Aaron,et al.  Learning Visual Behavior for Gesture AnalysisAndrew , 1995 .

[23]  Hsi-Jian Lee,et al.  Knowledge-guided visual perception of 3-D human gait from a single image sequence , 1992, IEEE Trans. Syst. Man Cybern..

[24]  Mubarak Shah,et al.  Matching motion trajectories using scale-space , 1993, Pattern Recognit..

[25]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[26]  A I Tew,et al.  A real-time gesture recognizer based on dynamic programming. , 1993, Journal of biomedical engineering.

[27]  William T. Freeman,et al.  Orientation Histograms for Hand Gesture Recognition , 1995 .

[28]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[29]  R. Nelson,et al.  Low level recognition of human motion (or how to get your man without finding his body parts) , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[30]  M. Turk,et al.  Eigenfaces for Recognition , 1991, Journal of Cognitive Neuroscience.

[31]  A. Dale Magoun,et al.  Decision, estimation and classification , 1989 .

[32]  T. Poggio,et al.  A network that learns to recognize three-dimensional objects , 1990, Nature.

[33]  Tomaso Poggio,et al.  Example Based Image Analysis and Synthesis , 1993 .

[34]  Irfan Essa,et al.  Tracking facial motion , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[35]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[36]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[37]  K. Rohr.  Towards model-based recognition of human movements in image sequences , 1994 .

[38]  Alex Pentland,et al.  Probabilistic visual learning for object detection , 1995, Proceedings of IEEE International Conference on Computer Vision.

[39]  John,et al.  On Comprehensive Visual Learning , 1994 .

[40]  James S. Lipscomb.  A trainable gesture recognizer , 1991, Pattern Recognit..

[41]  Alex Pentland,et al.  A vision system for observing and extracting facial action parameters , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Biing-Hwang Juang,et al.  Hidden Markov Models for Speech Recognition , 1991 .

[43]  Richard A. Bolt,et al.  Two-handed gesture in multi-modal natural dialog , 1992, UIST '92.

[44]  Takeo Kanade,et al.  DigitEyes: Vision-Based Human Hand Tracking , 1993 .

[45]  Kanti V. Mardia,et al.  Techniques for online gesture recognition on workstations , 1993, Image Vis. Comput..

[46]  Aaron F. Bobick,et al.  A state-based technique for the summarization and recognition of gesture , 1995, Proceedings of IEEE International Conference on Computer Vision.

[47]  A E Marble,et al.  Image processing system for interpreting motion in American Sign Language. , 1992, Journal of biomedical engineering.

[48]  Alan Wexelblat,et al.  A feature-based approach to continuous-gesture analysis , 1994 .

[49]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[50]  James W. Davis,et al.  GESTURE RECOGNITION , 2023, International Research Journal of Modernization in Engineering Technology and Science.