The Visual Analysis of Human Movement: A Survey

The ability to recognize humans and their activities by vision is key for a machine to interact intelligently and effortlessly with a human-inhabited environment. Because of many potentially important applications, “looking at people” is currently one of the most active application domains in computer vision. This survey identifies a number of promising applications and provides an overview of recent developments in this domain. The scope of this survey is limited to work on whole-body or hand motion; it does not include work on human faces. The emphasis is on discussing the various methodologies; they are grouped in 2-D approaches with or without explicit shape models and 3-D approaches. Where appropriate, systems are reviewed. We conclude with some thoughts about future directions.

[1]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[2]  D. Marr,et al.  Representation and recognition of the spatial organization of three-dimensional shapes , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[3]  Martin Herman,et al.  Understanding body postures of human stick figures , 1979 .

[4]  Norman I. Badler,et al.  Digital Representations of Human Movement , 1979, CSUR.

[5]  J. O'Rourke,et al.  Model-based image analysis of human motion using constraint propagation , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Aaron E. Rosenberg,et al.  Performance tradeoffs in dynamic time warping algorithms for isolated word recognition , 1980 .

[7]  Jake K. Aggarwal,et al.  Structure from Motion of Rigid and Jointed Objects , 1981, Artif. Intell..

[8]  David C. Hogg.  Model-based vision: a program to see a walking person , 1983, Image Vis. Comput..

[9]  Alan H. Barr,et al.  Global and local deformations of solid primitives , 1984, SIGGRAPH.

[10]  Koichiro Akita,et al.  Image sequence analysis of real world human motion , 1984, Pattern Recognit..

[11]  Shinichi Tamura,et al.  Recognition of sign language motion images , 1988, Pattern Recognit..

[12]  Mark W. Spong,et al.  Robot dynamics and control , 1989 .

[13]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[14]  Daniel Thalmann,et al.  Human Modeling and Animation , 1990 .

[15]  Bernd Neumann,et al.  On the Use of Motion Concepts for Top-Down Control in Traffic Scenes , 1990, ECCV.

[16]  Masanobu Yamamoto,et al.  Human motion analysis based on a robot arm model , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Yee-Hong Yang,et al.  Log-Tracker: an Attribute-Based Approach to Tracking Human Body Motion , 1991, Int. J. Pattern Recognit. Artif. Intell..

[18]  J. Sklansky,et al.  Segmentation of people in motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[19]  Takeshi Shakunaga.  Pose estimation of jointed structures , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[20]  Ramakant Nevatia,et al.  Description and tracking of moving articulated objects , 1992, [1992] Proceedings. 11th IAPR International Conference on Pattern Recognition.

[21]  A. C. Downton,et al.  Model-based image analysis for unconstrained human upper-body motion , 1992 .

[22]  Junji Yamato,et al.  Recognizing human action in time-sequential images using hidden Markov model , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Hsi-Jian Lee,et al.  Knowledge-guided visual perception of 3-D human gait from a single image sequence , 1992, IEEE Trans. Syst. Man Cybern..

[24]  A E Marble,et al.  Image processing system for interpreting motion in American Sign Language. , 1992, Journal of biomedical engineering.

[25]  Moving posture reconstruction from perspective projections of jointed figure motion , 1993 .

[26]  Alex Pentland,et al.  Space-time gestures , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Mubarak Shah,et al.  Matching motion trajectories using scale-space , 1993, Pattern Recognit..

[28]  Dimitris N. Metaxas,et al.  Shape and Nonrigid Motion Estimation Through Physics-Based Synthesis , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  Francisco J. Perales,et al.  A system for human motion matching between synthetic and real images based on a biomechanic graphical model , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[30]  Fumio Kishino,et al.  Human posture estimation from multiple images using genetic algorithm , 1994, Proceedings of 12th International Conference on Pattern Recognition.

[31]  R. Nelson,et al.  Low level recognition of human motion (or how to get your man without finding his body parts) , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[32]  Gang Xu,et al.  Understanding human motion patterns , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[33]  E. Adelson,et al.  Analyzing gait with spatiotemporal surfaces , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[34]  Hans-Hellmut Nagel,et al.  Association of Motion Verbs with Vehicle Movements Extracted from Dense Optical Flow Fields , 1994, ECCV.

[35]  Nigel Goddard,et al.  Incremental model-based discrimination of articulated movement from motion features , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[36]  R. Oka,et al.  Recognition of dexterous manipulations from time-varying images , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[37]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[38]  David C. Hogg,et al.  An Eecient Method for Contour Tracking Using Active Shape Models , 1994 .

[39]  David C. Hogg,et al.  An efficient method for contour tracking using active shape models , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[40]  Ramakant Nevatia,et al.  Description and tracking of moving articulated objects , 1994, Systems and Computers in Japan.

[41]  Alexander Geurtz.  Model based shape estimation , 1994, Signal Process..

[42]  K. Rohr.  Towards model-based recognition of human movements in image sequences , 1994 .

[43]  Jake K. Aggarwal,et al.  Articulated and elastic non-rigid motion: a review , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[44]  Arthur E. Chapman,et al.  Analysis and synthesis of human movement , 1994 .

[45]  Charles L. Wilson,et al.  Face Recognition Technology for Law Enforcement Applications , 1994 .

[46]  M. Rosenblum,et al.  Human emotion recognition from motion using a radial basis function network architecture , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[47]  Takeo Kanade,et al.  Visual Tracking of High DOF Articulated Structures: an Application to Human Hand Tracking , 1994, ECCV.

[48]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[49]  James W. Davis,et al.  GESTURE RECOGNITION , 2023, International Research Journal of Modernization in Engineering Technology and Science.

[50]  Francis K. H. Quek.  Eyes in the interface , 1995, Image Vis. Comput..

[51]  Thomas S. Huang,et al.  Vision based hand modeling and tracking for virtual teleconferencing and telecollaboration , 1995, Proceedings of IEEE International Conference on Computer Vision.

[52]  Yee-Hong Yang,et al.  First Sight: A Human Body Outline Labeling System , 1995, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Ioannis A. Kakadiaris,et al.  3D human body model acquisition from multiple views , 1995, Proceedings of IEEE International Conference on Computer Vision.

[54]  Aaron F. Bobick,et al.  Recognition of human body motion using phase space constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[55]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[56]  Aaron F. Bobick,et al.  A state-based technique for the summarization and recognition of gesture , 1995, Proceedings of IEEE International Conference on Computer Vision.

[57]  Pietro Perona,et al.  Monocular tracking of the human arm in 3D , 1995, Proceedings of IEEE International Conference on Computer Vision.

[58]  Alex Pentland,et al.  Real-time American Sign Language recognition from video using hidden Markov models , 1995 .

[59]  Takeo Kanade,et al.  Model-based tracking of self-occluding articulated objects , 1995, Proceedings of IEEE International Conference on Computer Vision.

[60]  Timothy F. Cootes,et al.  Active Shape Models-Their Training and Application , 1995, Comput. Vis. Image Underst..

[61]  Kiyoharu Aizawa,et al.  Model-based image coding advanced video coding techniques for very low bit-rate applications , 1995, Proc. IEEE.

[62]  Matthew Turk,et al.  Visual interaction with lifelike characters , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[63]  Alex Pentland,et al.  Real-time self-calibrating stereo person tracking using 3-D shape estimation from blob features , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[64]  Michael J. Swain,et al.  Gesture recognition using the Perseus architecture , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[65]  Ioannis A. Kakadiaris,et al.  Model-based estimation of 3D human motion with occlusion based on active multi-viewpoint selection , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[66]  I-Cheng Chang,et al.  Ribbon-based motion analysis of human body movements , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[67]  Kazuo Kyuma,et al.  Computer vision for computer games , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[68]  John R. Kender,et al.  Toward the use of gesture in traditional user interfaces , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[69]  Michael J. Black,et al.  Cardboard people: a parameterized model of articulated image motion , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[70]  David C. Hogg,et al.  Towards 3D hand tracking using a deformable model , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[71]  Jakub Segen,et al.  A camera-based system for tracking people in real time , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[72]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[73]  Larry S. Davis,et al.  3-D model-based tracking of humans in action: a multi-view approach , 1996, Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[74]  F. Perera.  Uncovering new clues to cancer risk. , 1996, Scientific American.

[75]  Jake K. Aggarwal,et al.  Tracking human motion using multiple cameras , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[76]  Robert J. Holt,et al.  Determining articulated motion from perspective views: a decomposition approach , 1997, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[77]  Alberto Broggi,et al.  Vision-Based Driving Assistance in Vehicles of the Future , 1997 .

[78]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[79]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[80]  Tomaso A. Poggio,et al.  Pedestrian detection using wavelet templates , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[81]  Ulrich Kressel,et al.  Tracking non-rigid, moving objects based on color cluster flow , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[82]  Dariu Mihai Gavrila,et al.  Vision-based 3-D tracking of humans in action , 1998 .

[83]  Frank Lindner,et al.  Autonomous Driving approaches Downtown , 1999 .