Gait analysis, modelling, and comparison from unconstrained walks and viewpoints : view-rectification of body-part trajectories from monocular video sequences

Gait analysis, modelling and comparison using computer vision algorithms has recently attracted much attention for medical and surveillance applications. Analyzing and modelling a person’s gait with computer vision algorithms has indeed some interesting advantages over more traditional biometrics. For instance, gait can be analyzed and modelled at a distance by observing the person with a camera, which means that no markers or sensors have to be worn by the person. Moreover, gait analysis and modelling using computer vision algorithms does not require the cooperation of the observed people, which thus allows for using gait as a biometric in surveillance applications. Current gait analysis and modelling approaches have however severe limitations. For instance, several approaches require a side view of the walks since this viewpoint is optimal for gait analysis and modelling. Most approaches also require the walks to be observed far enough from the camera in order to avoid perspective distortion effects that would badly affect the resulting gait analyses and models. Moreover, current approaches do not allow for changes in walk direction and in walking speed, which greatly constraints the walks that can be analyzed and modelled in medical and surveillance applications. The approach proposed in this thesis aims at performing gait analysis, modelling and comparison from unconstrained walks and viewpoints in medical and surveillance applications. The proposed approach mainly consists in a novel view-rectification method that generates a fronto-parallel viewpoint (side view) of the imaged trajectories of body parts. The view-rectification method is based on a novel walk model that uses projective geometry to provide the spatio-temporal links between the body-part positions in the scene and their corresponding positions in the images. The head and the feet are the only body parts that are relevant for the proposed approach. They are automatically localized and tracked in monocular video sequences using a novel body parts tracking algorithm. Gait analysis is performed by a novel method that extracts standard

[1]  Rama Chellappa,et al.  Gait Analysis for Human Identification , 2003, AVBPA.

[2]  R. Begg,et al.  Minimum foot clearance during walking: strategies for the minimisation of trip-related falls. , 2007, Gait & posture.

[3]  Yun-Hong Wang,et al.  Human gait recognition based on X-T plane energy images , 2007, 2007 International Conference on Wavelet Analysis and Pattern Recognition.

[4]  Simone Calderara,et al.  Reliable background suppression for complex scenes , 2006, VSSN '06.

[5]  Dimitris N. Metaxas,et al.  Human gait recognition at sagittal plane , 2007, Image Vis. Comput..

[6]  Mark S. Nixon,et al.  Gait recognition by dynamic cues , 2008, 2008 19th International Conference on Pattern Recognition.

[7]  Heba M. Lakany,et al.  Extracting a diagnostic gait signature , 2008, Pattern Recognit..

[8]  Rama Chellappa,et al.  Video synthesis of arbitrary views for approximately planar scenes , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[9]  Zhijing Liu,et al.  Fusion of Face and Gait for Human Recognition in Video Sequences , 2009, 2009 International Conference on Information Technology and Computer Science.

[10]  Liang Wang,et al.  Abnormal Walking Gait Analysis Using Silhouette-Masked Flow Histograms , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Robert Bergevin,et al.  Computing View-normalized Body Parts Trajectories , 2007, Fourth Canadian Conference on Computer and Robot Vision (CRV '07).

[12]  Alfred D. Grant.  Gait Analysis: Normal and Pathological Function , 2010 .

[13]  Robert Bergevin,et al.  Towards view-invariant gait modeling: Computing view-normalized body part trajectories , 2009, Pattern Recognit..

[14]  Mubarak Shah,et al.  Establishing motion correspondence , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  Byung-Uk Choi,et al.  Canonical View Synthesis for Gait Recognition , 2007, 2007 Frontiers in the Convergence of Bioscience and Information Technologies.

[16]  Adam Prügel-Bennett,et al.  Automatic gait recognition using area-based metrics , 2003 .

[17]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[18]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Amit K. Roy-Chowdhury,et al.  A study on view-insensitive gait recognition , 2005, IEEE International Conference on Image Processing 2005.

[20]  Chiraz Ben Abdelkader.  Motion-Based Recognition of People in EigenGait Space , 2002 .

[21]  Robert Bergevin,et al.  Body tracking in human walk from monocular video sequences , 2005, The 2nd Canadian Conference on Computer and Robot Vision (CRV'05).

[22]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[23]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Ethem Alpaydin,et al.  Introduction to Machine Learning (Adaptive Computation and Machine Learning) , 2004 .

[25]  Christopher Kirtley,et al.  Clinical Gait Analysis: Theory and Practice , 2006 .

[26]  Robert Bergevin,et al.  Human gait characteristics from unconstrained walks and viewpoints , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[27]  Saeid Rahati,et al.  Gait Recognition Using Wavelet Transform , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[28]  David J. Hewson,et al.  The PARAChute Project: Remote Monitoring of Posture and Gait for Fall Prevention , 2007, EURASIP J. Adv. Signal Process..

[29]  Haiping Lu,et al.  A Full-Body Layered Deformable Model for Automatic Model-Based Gait Recognition , 2008, EURASIP J. Adv. Signal Process..

[30]  Janne Heikkilä,et al.  Geometric Camera Calibration Using Circular Control Points , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Mark S. Nixon,et al.  Covariate Analysis for View-Point Independent Gait Recognition , 2009, ICB.

[32]  Seong-Whan Lee,et al.  Reconstruction of 3D Human Body Pose for Gait Recognition , 2006, ICB.

[33]  John N. Carter,et al.  Towards pose invariant gait reconstruction , 2005, IEEE International Conference on Image Processing 2005.

[34]  Rabia Jafri,et al.  Fusion of Face and Gait for Automatic Human Recognition , 2008, Fifth International Conference on Information Technology: New Generations (itng 2008).

[35]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Mark S. Nixon,et al.  Human Perambulation as a Self Calibrating Biometric , 2007, AMFG.

[37]  Mark S. Nixon,et al.  On a Large Sequence-Based Human Gait Database , 2004 .

[38]  Z. Liu,et al.  Simplest representation yet for gait recognition: averaged silhouette , 2004, ICPR 2004.

[39]  E. Ayyappa.  Normal Human Locomotion, Part 1: Basic Concepts and Terminology , 1997 .

[40]  Liu Ming,et al.  Identification of Individual Walking Patterns Using Gait Acceleration , 2007, 2007 1st International Conference on Bioinformatics and Biomedical Engineering.

[41]  Nanning Zheng,et al.  Gait History Image: A Novel Temporal Template for Gait Recognition , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[42]  Dimitrios Hatzinakos,et al.  Gait recognition using linear time normalization , 2006, Pattern Recognit..

[43]  Stefan Roth,et al.  People-tracking-by-detection and people-detection-by-tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Euntai Kim,et al.  A New Gait Representation for Human Identification: Mass Vector , 2007, 2007 2nd IEEE Conference on Industrial Electronics and Applications.

[45]  Michael W. Whittle,et al.  Gait Analysis: An Introduction , 1986 .

[46]  Bailing Zhang,et al.  Gait Classificaiton in Children with Cerebral Palsy by Bayesian Approach , 2007 .

[47]  Maricor N. Soriano,et al.  Compact time-independent pattern representation of entire human gait cycle for tracking of gait irregularities , 2010, Pattern Recognit. Lett..

[48]  L Sloman,et al.  Gait patterns of depressed patients and normal subjects. , 1982, The American journal of psychiatry.

[49]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[50]  A.H. Khandoker,et al.  A Wavelet-Based Approach for Screening Falls Risk in the Elderly using Support Vector Machines , 2006, 2006 Fourth International Conference on Intelligent Sensing and Information Processing.

[51]  Mark S. Nixon,et al.  Exploratory factor analysis of gait recognition , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[52]  Nikolaos V. Boulgouris,et al.  Human gait recognition based on matching of body components , 2007, Pattern Recognit..

[53]  Yangsheng Xu,et al.  Gait Modeling for Human Identification , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[54]  Tieniu Tan,et al.  People tracking based on motion model and motion constraints with automatic initialization , 2004, Pattern Recognit..

[55]  Jesús Martínez del Rincón,et al.  A spatio-temporal 2D-models framework for human pose recovery in monocular sequences , 2008, Pattern Recognit..

[56]  Aaron F. Bobick,et al.  A Multi-view Method for Gait Recognition Using Static Body Parameters , 2001, AVBPA.

[57]  J. Little,et al.  Recognizing People by Their Gait: The Shape of Motion , 1998 .

[58]  Bir Bhanu,et al.  Statistical feature fusion for gait-based human recognition , 2004, CVPR 2004.

[59]  J. Cutting,et al.  Recognizing friends by their walk: Gait perception without familiarity cues , 1977 .

[60]  M. Nixon,et al.  Model-Based Gait Enrolment in Real-World Imagery , 2003 .

[61]  Nigel J. B. McFarlane,et al.  Segmentation and tracking of piglets in images , 1995, Machine Vision and Applications.

[62]  Jiwen Lu,et al.  Gait Recognition via Independent Component Analysis Based on Support Vector Machine and Neural Network , 2005, ICNC.

[63]  D. Hatzinakos,et al.  Gait recognition: a challenging signal processing technology for biometric identification , 2005, IEEE Signal Processing Magazine.

[64]  Rongchun Zhao,et al.  Automatic Gait Recognition using Dynamic Variance Features , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[65]  Raymond S. T. Lee,et al.  Silhouette Spatio-temporal Spectrum (SStS) for Gait-Based Human Recognition , 2005, ICAPR.

[66]  Dongdong Nie,et al.  Gait Recognition at a Distance Based on Energy Deviation Image , 2007, 2007 1st International Conference on Bioinformatics and Biomedical Engineering.

[67]  R. Bergevin,et al.  Progressive Human Skeleton Fitting , 2003 .

[68]  Dimitrios Hatzinakos,et al.  Gait recognition using dynamic time warping , 2004, IEEE 6th Workshop on Multimedia Signal Processing, 2004..

[69]  Murat Ekinci.  Gait Recognition Using Multiple Projections , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[70]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[71]  W. Eric L. Grimson,et al.  Gait Appearance for Recognition , 2002, Biometric Authentication.

[72]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[73]  Tieniu Tan,et al.  Gait analysis for human identification in frequency domain , 2004, Third International Conference on Image and Graphics (ICIG'04).

[74]  S. Stevenage,et al.  Visual analysis of gait as a cue to identity , 1999 .

[75]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[76]  N. Spencer.  Pose invariant gait analysis and reconstruction , 2005 .

[77]  Larry S. Davis,et al.  View-invariant Estimation of Height and Stride for Gait Recognition , 2002, Biometric Authentication.

[78]  Mark S. Nixon,et al.  Model-Based Feature Extraction for Gait Analysis and Recognition , 2007, MIRAGE.

[79]  Robert Bergevin,et al.  Computing and evaluating view-normalized body part trajectories , 2009, Image Vis. Comput..

[80]  Robert P. W. Duin,et al.  A multi-classifier for grading knee osteoarthritis using gait analysis , 2010, Pattern Recognit. Lett..

[81]  Georgios Tziritas,et al.  Recognition and Tracking of the Members of a Moving Human Body , 2004, AMDO.

[82]  Ahmed M. Elgammal,et al.  Towards Scalable View-Invariant Gait Recognition: Multilinear Analysis for Gait , 2005, AVBPA.

[83]  Robert T. Collins,et al.  Silhouette-based human identification from body shape and gait , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[84]  M. P. Murray.  Gait as a total pattern of movement. , 1967, American journal of physical medicine.

[85]  Shaogang Gong,et al.  Feature selection on Gait Energy Image for human identification , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[86]  Qing Wang,et al.  A Novel Gait Recognition Method Via Fusing Shape and Kinematics Features , 2006, ISVC.

[87]  Wei Jia,et al.  Gait Recognition Using Hough Transform and Principal Component Analysis , 2009, ICIC.

[88]  Rafael C. González,et al.  Local Determination of a Moving Contrast Edge , 1985, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[89]  Jiwen Lu,et al.  Uncorrelated discriminant simplex analysis for view-invariant gait signal computing , 2010, Pattern Recognit. Lett..

[90]  Berthold K. P. Horn,et al.  Determining Optical Flow , 1981, Other Conferences.

[91]  Rama Chellappa,et al.  Towards a view invariant gait recognition algorithm , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[92]  Robert Bergevin,et al.  Trajectories normalization for viewpoint invariant gait recognition , 2008, 2008 19th International Conference on Pattern Recognition.

[93]  Mark S. Nixon,et al.  Automatic extraction and description of human gait models for recognition purposes , 2003, Comput. Vis. Image Underst..

[94]  K. S. Arun,et al.  Least-Squares Fitting of Two 3-D Point Sets , 1987, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[95]  Chung-Lin Huang,et al.  Gait Analysis For Human Identification Through Manifold Learning and HMM , 2007, 2007 IEEE Workshop on Motion and Video Computing (WMVC'07).

[96]  M.S. Nixon,et al.  Front-view Gait Recognition , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[97]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[98]  S. Enokida,et al.  A Predictive Model for Gait Recognition , 2006, 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference.

[99]  Hua Li,et al.  3D gait recognition using multiple cameras , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[100]  Tieniu Tan,et al.  Fusion of static and dynamic body biometrics for gait recognition , 2004, Proceedings Ninth IEEE International Conference on Computer Vision.

[101]  Jorge S. Marques,et al.  Tracking the Human Body Using Multiple Predictors , 2002, AMDO.

[102]  Zoran Zivkovic,et al.  Improved adaptive Gaussian mixture model for background subtraction , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[103]  Marc Parizeau,et al.  MONNET: Monitoring Pedestrians with a Network of Loosely-Coupled Cameras , 2006, 18th International Conference on Pattern Recognition (ICPR'06).