Towards 3-D model-based tracking and recognition of human movement: a multi-view approach

In this paper we describe our work on 3-D model-based tracking and recognition of human movement from real images. Our system has two major components. The rst component takes real image sequences acquired from multiple views and recovers the 3-D body pose at each time instant. The pose-recovery problem is formulated as a search problem and entails nding the pose parameters of a graphical human model for which its synthesized appearance is most similar to the actual appearance of the real human in the multi-view images. Currently, we use a best-rst search technique and chamfer matching as a fast similarity measure between synthesized and real edge images. The second component of our system deals with the representation and recognition of human movement patterns. The recognition of human movement patterns is considered as a classiication problem involving the matching of a test sequence with several reference sequences representing prototypical activities. A variation of dynamic time-warping is used to match movement patterns using 3-D joint angles as features. We illustrate our approach on real data acquired simultaneously from three views and data derived from stereo Moving Light Displays with diierent types of hand-gestures.

[1]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[2]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[3]  Robert C. Bolles,et al.  Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching , 1977, IJCAI.

[4]  D. Marr,et al.  Representation and recognition of the spatial organization of three-dimensional shapes , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[5]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[6]  Koichiro Akita,et al.  Image sequence analysis of real world human motion , 1984, Pattern Recognit..

[7]  Michael Jenkin,et al.  Tracking three-dimensional moving light displays , 1986, Workshop on Motion.

[8]  Yee-Hong Yang,et al.  Log-Tracker: an Attribute-Based Approach to Tracking Human Body Motion , 1991, Int. J. Pattern Recognit. Artif. Intell..

[9]  Dimitris N. Metaxas,et al.  Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  A. C. Downton,et al.  Model-based image analysis for unconstrained human upper-body motion , 1992 .

[11]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[12]  Moving posture reconstruction from perspective projections of jointed figure motion , 1993 .

[13]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Mubarak Shah,et al.  Matching motion trajectories using scale-space , 1993, Pattern Recognit..

[15]  Takeo Kanade,et al.  DigitEyes: Vision-Based Human Hand Tracking , 1993 .

[16]  Francisco J. Perales,et al.  A system for human motion matching between synthetic and real images based on a biomechanic graphical model , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[17]  Fumio Kishino,et al.  Human posture estimation from multiple images using genetic algorithm , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[18]  R. Nelson,et al.  Low level recognition of human motion (or how to get your man without finding his body parts) , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[19]  Richard Szeliski,et al.  Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares , 1994, J. Vis. Commun. Image Represent..

[20]  Nigel Goddard,et al.  Incremental model-based discrimination of articulated movement from motion features , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[21]  R. Oka,et al.  Recognition of dexterous manipulations from time-varying images , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[22]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[23]  K. Rohr.  Towards model-based recognition of human movements in image sequences , 1994 .

[24]  M. K. Leung,et al.  A model based approach to labelling human body outlines , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[25]  Fumio Kishino,et al.  Human Posture Detection from Trinocular Images Using Genetic Algorithm : Posture Estimation of Upper Half Bodies and Fingers , 1994 .

[26]  M. Rosenblum,et al.  Human emotion recognition from motion using a radial basis function network architecture , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[27]  Lee W. Campbell.  Recognizing classical ballet steps using plase space constraints , 1994 .

[28]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..