Gait analysis and recognition for automated visual surveillance

Human motion analysis has received a great attention from researchers in the last decade due to its potential use in different applications such as automated visual surveillance. This field of research focuses on the perception and recognition of human activities, including people identification. We explore a new approach for walking pedestrian detection in an unconstrained outdoor environment. The proposed algorithm is based on gait motion as the rhythm of the footprint pattern of walking people is considered the stable and characteristic feature for the classification of moving objects. The novelty of our approach is motivated by the latest research for people identification using gait. The experimental results confirmed the robustness of our method to discriminate between single walking subject, groups of people and vehicles with a successful detection rate of 100%. Furthermore, the results revealed the potential of our method to extend visual surveillance systems to recognize walking people. Furthermore, we propose a new approach to extract human joints (vertex positions) using a model-based method. The spatial templates describing the human gait motion are produced via gait analysis performed on data collected from manual labeling. The Elliptic Fourier Descriptors are used to represent the motion models in a parametric form. The heel strike data is exploited to reduce the dimensionality of the parametric models. People walk normal to the viewing plane, as major gait information is available in a sagittal view. The ankle, knee and hip joints are successfully extracted with high accuracy for indoor and outdoor data. In this way, we have established a baseline analysis which can be deployed in recognition, marker-less analysis and other areas. The experimental results confirmed the robustness of the model-based approach to recognise walking subjects with a correct classification rate of 95% using purely the dynamic features derived from the joint motion. Therefore, this confirms the early psychological theories claiming that the discriminative features for motion perception and people recognition are embedded in gait kinematics. Furthermore, to quantify the intrusive nature of gait recognition we explore the effects of the different covariate factors on the performance of gait recognition. The covariate factors include footwear, clothing, carrying conditions and walking speed. As far as the author can determine, this is the first major study of its kind in this field to analyse the covariate factors using a model-based method.

[1]  Mark S. Nixon,et al.  Model-Based Feature Extraction for Gait Analysis and Recognition , 2007, MIRAGE.

[2]  Nigel Goddard,et al.  The Perception of Articulated Motion: Recognizing Moving Light Displays , 1992 .

[3]  Rama Chellappa,et al.  A hidden Markov model based framework for recognition of humans from gait sequences , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[4]  Mark S. Nixon,et al.  Automatic extraction and description of human gait models for recognition purposes , 2003, Comput. Vis. Image Underst..

[5]  W. Eric L. Grimson,et al.  Gait analysis for recognition and classification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[6]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[7]  Pavel Pudil,et al.  Introduction to Statistical Pattern Recognition , 2006 .

[8]  A. B. Drought,et al.  WALKING PATTERNS OF NORMAL MEN. , 1964, The Journal of bone and joint surgery. American volume.

[9]  John Daugman,et al.  High Confidence Visual Recognition of Persons by a Test of Statistical Independence , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  J. Ohya,et al.  Real-time estimation of human body posture from monocular thermal images , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[11]  Aaron F. Bobick,et al.  Gait recognition from time-normalized joint-angle trajectories in the walking plane , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[12]  J. Cutting,et al.  Gait Perception as an Example of How We May Perceive Events , 1981 .

[13]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[14]  J. Little,et al.  Recognizing People by Their Gait: The Shape of Motion , 1998 .

[15]  Dorin Comaniciu,et al.  Real-time tracking of non-rigid objects using mean shift , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[16]  J. Sklansky,et al.  Segmentation of people in motion , 1991, Proceedings of the IEEE Workshop on Visual Motion.

[17]  G P Bingham,et al.  Dynamics and the orientation of kinematic forms in visual event recognition. , 1995, Journal of experimental psychology. Human perception and performance.

[18]  Andrew Blake,et al.  Articulated body motion capture by annealed particle filtering , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[19]  John N. Carter,et al.  Towards pose invariant gait reconstruction , 2005, IEEE International Conference on Image Processing 2005.

[20]  David A. Winter,et al.  Human balance and posture control during standing and walking , 1995 .

[21]  J. Cutting,et al.  Recognizing the gender of walkers from point-lights mounted on ankles: Some second thoughts , 1978 .

[22]  Tieniu Tan,et al.  A new attempt to gait-based human identification , 2002, Object recognition supported by user interaction for service robots.

[23]  J. L. Roux An Introduction to the Kalman Filter , 2003 .

[24]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[25]  Mark S. Nixon,et al.  Parameterizing Arbitrary Shapes via Fourier Descriptors for Evidence-Gathering Extraction , 1998, Comput. Vis. Image Underst..

[26]  R. Blake,et al.  Perception of human motion. , 2007, Annual review of psychology.

[27]  W. Dittrich Action Categories and the Perception of Biological Motion , 1993, Perception.

[28]  M. Woollacott,et al.  Balance control during walking in the older adult: research and its implications. , 1997, Physical therapy.

[29]  T Chau,et al.  A review of analytical techniques for gait data. Part 1: Fuzzy, statistical and fractal methods. , 2001, Gait & posture.

[30]  V. F. Leavers,et al.  Which Hough transform , 1993 .

[31]  David Kenneth Wagg Local and global models for articulated motion analysis , 2006 .

[32]  Tieniu Tan,et al.  Fusion of static and dynamic body biometrics for gait recognition , 2003, IEEE Transactions on Circuits and Systems for Video Technology.

[33]  Mark S. Nixon,et al.  Bias Error Analysis of the Generalised Hough Transform , 2004, Journal of Mathematical Imaging and Vision.

[34]  Josef Kittler,et al.  Pattern recognition : a statistical approach , 1982 .

[35]  P. Jonathon Phillips Human identification technical challenges , 2002, Proceedings. International Conference on Image Processing.

[36]  Mark S. Nixon,et al.  Gait Verification Using Probabilistic Methods , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[37]  M. P. Murray Gait as a total pattern of movement. , 1967, American journal of physical medicine.

[38]  B. Nigg,et al.  Changing the texture of footwear can alter gait patterns. , 2005, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[39]  R. Chellappa,et al.  Human Identification Based on Gait (The Kluwer International Series on Biometrics) , 2005 .

[40]  ChewYean Yam,et al.  On the relationship of human walking and running: automatic person identification by gait , 2002, Object recognition supported by user interaction for service robots.

[41]  Mark S. Nixon,et al.  Bayesian statistics and modelling , 2020, Nature Reviews Methods Primers.

[42]  Anil K. Jain,et al.  39 Dimensionality and sample size considerations in pattern recognition practice , 1982, Classification, Pattern Recognition and Reduction of Dimensionality.

[43]  Carlos Orrite-Uruñuela,et al.  Human Recognition by Gait Analysis Using Neural Networks , 2002, ICANN.

[44]  W. E. Egan,et al.  Postural sway of subjects wearing the U.S. Army chemical protective ensemble after functional activity. , 2001, Aviation, space, and environmental medicine.

[45]  Gösta H. Granlund,et al.  Fourier Preprocessing for Hand Print Character Recognition , 1972, IEEE Transactions on Computers.

[46]  Jeffrey K. Uhlmann,et al.  New extension of the Kalman filter to nonlinear systems , 1997, Defense, Security, and Sensing.

[47]  Michael Isard,et al.  CONDENSATION—Conditional Density Propagation for Visual Tracking , 1998, International Journal of Computer Vision.

[48]  A. David Marshall,et al.  A Hierarchical Model of Dynamics for Tracking People with a Single Video Camera , 2000, BMVC.

[49]  J B King,et al.  Gait Analysis. An Introduction , 1992 .

[50]  L. Middleton,et al.  Fusion of dynamic and static features for gait recognition over time , 2005, 2005 7th International Conference on Information Fusion.

[51]  Chung-Lin Huang,et al.  Hand gesture recognition using a real-time tracking method and hidden Markov models , 2003, Image Vis. Comput..

[52]  Aaron F. Bobick,et al.  A Multi-view Method for Gait Recognition Using Static Body Parameters , 2001, AVBPA.

[53]  R. Kleissen,et al.  Electromyography in the biomechanical analysis of human movement and its clinical application. , 1998, Gait & posture.

[54]  Aaron F. Bobick,et al.  Recognition of human body motion using phase space constraints , 1995, Proceedings of IEEE International Conference on Computer Vision.

[55]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[56]  Mubarak Shah,et al.  Motion-based recognition a survey , 1995, Image Vis. Comput..

[57]  Larry S. Davis,et al.  Stride and cadence as a biometric in automatic person identification and verification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[58]  Mark S. Nixon,et al.  Automatic Recognition by Gait , 2006, Proceedings of the IEEE.

[59]  Mark S. Nixon,et al.  Automatic Gait Recognition via Fourier Descriptors of Deformable Objects , 2003, AVBPA.

[60]  Mark S. Nixon,et al.  Advances in automatic gait recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[61]  Mubarak Shah,et al.  Tracking and Object Classification for Automated Surveillance , 2002, ECCV.

[62]  Mark S. Nixon,et al.  On a Large Sequence-Based Human Gait Database , 2004 .

[63]  Hyeonjoon Moon,et al.  The FERET evaluation methodology for face-recognition algorithms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[64]  J. Knapik,et al.  Load carriage using packs: a review of physiological, biomechanical and medical aspects. , 1996, Applied ergonomics.

[65]  Bobby Bodenheimer,et al.  The Process of Motion Capture: Dealing with the Data , 1997, Computer Animation and Simulation.

[66]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[67]  Jake K. Aggarwal,et al.  Problems, ongoing research and future directions in motion research , 2003, Machine Vision and Applications.

[68]  Tieniu Tan,et al.  Silhouette Analysis-Based Gait Recognition for Human Identification , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[69]  Rita Cucchiara,et al.  Detecting Moving Objects, Ghosts, and Shadows in Video Streams , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[70]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[71]  Guojun Lu,et al.  Evaluation of similarity measurement for image retrieval , 2003, International Conference on Neural Networks and Signal Processing, 2003. Proceedings of the 2003.

[72]  N. Troje,et al.  Person identification from biological motion: Effects of structural and kinematic cues , 2005, Perception & psychophysics.

[73]  Montse Pardàs,et al.  Robust Tracking and Object Classification Towards Automated Video Surveillance , 2004, ICIAR.

[74]  Tieniu Tan,et al.  People tracking based on motion model and motion constraints with automatic initialization , 2004, Pattern Recognit..

[75]  R. Luukkonen,et al.  Protective equipment affects balance abilities differently in younger and older firefighters. , 2003, Aviation, space, and environmental medicine.

[76]  Richard A. Brand,et al.  The biomechanics and motor control of human gait: Normal, elderly, and pathological , 1992 .

[77]  A. Venetsanopoulos,et al.  A color segmentation scheme for object-based video coding , 1998, 1998 IEEE Symposium on Advances in Digital Filtering and Signal Processing. Symposium Proceedings (Cat. No.98EX185).

[78]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[79]  Hironobu Fujiyoshi,et al.  Moving target classification and tracking from real-time video , 1998, Proceedings Fourth IEEE Workshop on Applications of Computer Vision. WACV'98 (Cat. No.98EX201).

[80]  Paul H. Lewis,et al.  Extracting moving shapes by evidence gathering , 2002, Pattern Recognit..

[81]  Eadweard Muybridge,et al.  Muybridge's Complete human and animal locomotion : all 781 plates from the 1887 Animal locomotion , 1979 .

[82]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[83]  Dana H. Ballard,et al.  Generalizing the Hough transform to detect arbitrary shapes , 1981, Pattern Recognit..

[84]  Larry S. Davis,et al.  Person identification using automatic height and stride estimation , 2002, Object recognition supported by user interaction for service robots.

[85]  Niels Lynnerup,et al.  Person identification by gait analysis and photogrammetry. , 2005, Journal of forensic sciences.

[86]  Sudeep Sarkar,et al.  Statistical Motion Model Based on the Change of Feature Relationships: Human Gait-Based Recognition , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[87]  Mark S. Nixon,et al.  Extraction and Description of Moving Human Body by Periodic Motion Analysis , 2002, Computers and Their Applications.

[88]  Harriet A Allen,et al.  Visual mechanisms of motion analysis and motion perception. , 2004, Annual review of psychology.

[89]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[90]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[91]  J.K. Aggarwal,et al.  Correspondence processes in dynamic scene analysis , 1981, Proceedings of the IEEE.

[92]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[93]  Aaron F. Bobick,et al.  Gait recognition using static, activity-specific parameters , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[94]  Jake K. Aggarwal,et al.  Nonrigid Motion Analysis: Articulated and Elastic Motion , 1998, Comput. Vis. Image Underst..

[95]  Michael Gleicher,et al.  Animation from observation: Motion capture and motion editing , 1999, COMG.

[96]  Aaron F. Bobick,et al.  Modelling the effects of walking speed on appearance-based gait recognition , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[97]  W. T. Dempster,et al.  Properties of body segments based on size and weight , 1967 .

[98]  Anil K. Jain,et al.  Feature Selection: Evaluation, Application, and Small Sample Performance , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[99]  Sudeep Sarkar,et al.  The gait identification challenge problem: data sets and baseline algorithm , 2002, Object recognition supported by user interaction for service robots.

[100]  Deanna J. Fish,et al.  Walking Impediments and Gait Inefficiencies in the CVA Patient , 1999 .

[101]  Mark S. Nixon,et al.  Using Gait as a Biometric, via Phase-weighted Magnitude Spectra , 1997, AVBPA.

[102]  Charles R. Giardina,et al.  Elliptic Fourier features of a closed contour , 1982, Comput. Graph. Image Process..

[103]  M. Nixon,et al.  Human gait recognition in canonical space using temporal templates , 1999 .

[104]  Mark S. Nixon,et al.  Automated person recognition by walking and running via model-based approaches , 2004, Pattern Recognit..

[105]  L Sloman,et al.  Gait patterns of depressed patients and normal subjects. , 1982, The American journal of psychiatry.

[106]  G. Baudat,et al.  Generalized Discriminant Analysis Using a Kernel Approach , 2000, Neural Computation.

[107]  G. Johansson Visual perception of biological motion and a model for its analysis , 1973 .

[108]  Bor-Chen Kuo,et al.  Nonparametric weighted feature extraction for classification , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[109]  Ramakant Nevatia,et al.  Description and tracking of moving articulated objects , 1994, Systems and Computers in Japan.

[110]  Koichiro Akita,et al.  Image sequence analysis of real world human motion , 1984, Pattern Recognit..

[111]  S. Stevenage,et al.  Visual analysis of gait as a cue to identity , 1999 .

[112]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[113]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[114]  R. Dobbs,et al.  Is this walk normal? , 1993, Age and ageing.

[115]  D. Hatzinakos,et al.  Gait recognition: a challenging signal processing technology for biometric identification , 2005, IEEE Signal Processing Magazine.

[116]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[117]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[118]  A. N. Rajagopalan,et al.  Gait-based recognition of humans using continuous HMMs , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[119]  Mark S. Nixon,et al.  On automated model-based extraction and analysis of gait , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[120]  Nigel Goddard,et al.  Incremental model-based discrimination of articulated movement from motion features , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[121]  Adam Prügel-Bennett,et al.  Automatic gait recognition using area-based metrics , 2003 .

[122]  Aaron F. Bobick,et al.  Performance Analysis of Time-Distance Gait Parameters under Different Speeds , 2003, AVBPA.

[123]  R. L. Attwells,et al.  Influence of carrying heavy loads on soldiers' posture, movements and gait , 2006, Ergonomics.

[124]  Tieniu Tan,et al.  Automatic gait recognition based on statistical shape analysis , 2003, IEEE Trans. Image Process..

[125]  Nixon,et al.  Feature Extraction & Image Processing , 2008 .

[126]  Pavel Paclík,et al.  Adaptive floating search methods in feature selection , 1999, Pattern Recognit. Lett..

[127]  Joachim Denzler,et al.  Model based extraction of articulated objects in image sequences for gait analysis , 1997, Proceedings of International Conference on Image Processing.

[128]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[129]  Anil K. Jain,et al.  Algorithms for feature selection: An evaluation , 1996, Proceedings of 13th International Conference on Pattern Recognition.