Fusion of spatial-temporal and kinematic features for gait recognition with deterministic learning

We present a gait recognition method based on the fusion of different features.Spatial-temporal and kinematic features can fused for human identification.We show good recognition performance on four widely used gait databases. For obtaining optimal performance, as many informative cues as possible should be involved in the gait recognition algorithm. This paper describes a gait recognition algorithm by combining spatial-temporal and kinematic gait features. For each walking sequence, the binary silhouettes are characterized with four time-varying spatial-temporal parameters, including three lower limbs silhouette widths and one holistic silhouette area. Using deterministic learning algorithm, spatial-temporal gait features can be represented as the gait dynamics underlying the trajectories of lower limbs silhouette widths and holistic silhouette area, which can implicitly reflect the temporal changes of silhouette shape. In addition, a model-based method is proposed to extract joint-angle trajectories of lower limbs. Kinematic gait features can be represented as the gait dynamics underlying the trajectories of joint angles, which can represent the temporal changes of body structure and dynamics. Both spatial-temporal and kinematic cues can be used separately for gait recognition using smallest error principle. They are fused on the decision level using different combination rules to improve the gait recognition performance. The fusion of two different kinds of features provides a comprehensive characterization of gait dynamics, which is not sensitive to the walking conditions variation. The proposed method can still achieve superior performance when the testing walking conditions are different from the corresponding training conditions. Experimental results show that encouraging recognition accuracy can be achieved on five public gait databases: CASIA-B, CASIA-C, TUM GAID, OU-ISIR, USF HumanID.

[1]  Ricardo Matsumura de Araújo,et al.  Person Identification Using Anthropometric and Gait Data from Kinect Sensor , 2015, AAAI.

[2]  Wei Xiong,et al.  Active energy image plus 2DLPP for gait recognition , 2010, Signal Process..

[3]  Wei Zeng,et al.  Silhouette-Based Gait Recognition via Deterministic Learning , 2013, BICS.

[4]  Shamik Sural,et al.  Frontal Gait Recognition From Incomplete Sequences Using RGB-D Camera , 2014, IEEE Transactions on Information Forensics and Security.

[5]  Arun Ross,et al.  Gait curves for human recognition, backpack detection, and silhouette correction in a nighttime environment , 2010, Defense + Commercial Sensing.

[6]  Cong Wang,et al.  Human gait recognition based on deterministic learning through multiple views fusion , 2016, Pattern Recognit. Lett..

[7]  Chris J. Harris,et al.  Extracting Gait Signatures based on Anatomical Knowledge , 2002 .

[8]  Tieniu Tan,et al.  Orthogonal Diagonal Projections for Gait Recognition , 2007, 2007 IEEE International Conference on Image Processing.

[9]  Alan W. C. Tan,et al.  Gait probability image: An information-theoretic model of gait representation , 2014, J. Vis. Commun. Image Represent..

[10]  Tieniu Tan,et al.  Recognizing Night Walkers Based on One Pseudoshape Representation of Gait , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Z. Liu,et al.  Simplest representation yet for gait recognition: averaged silhouette , 2004, ICPR 2004.

[12]  Mark S. Nixon,et al.  On including quality in applied automatic gait recognition , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[13]  Alan W. C. Tan,et al.  Gait recognition via optimally interpolated deformable contours , 2013, Pattern Recognit. Lett..

[14]  Rudolf Fleischer,et al.  Low-Resolution Gait Recognition , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Tardi Tjahjadi,et al.  Robust view-invariant multiscale gait recognition , 2015, Pattern Recognit..

[16]  Mark S. Nixon,et al.  Automated Markerless Analysis of Human Gait Motion for Recognition and Classification , 2011 .

[17]  Chang-Tsun Li,et al.  A robust speed-invariant gait recognition system for walker and runner identification , 2013, 2013 International Conference on Biometrics (ICB).

[18]  Aaron F. Bobick,et al.  Gait recognition using static, activity-specific parameters , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[19]  Claudia Linnhoff-Popien,et al.  Gait Recognition with Kinect , 2012 .

[20]  Qiang Wu,et al.  Automatic Gait Recognition Using Weighted Binary Pattern on Video , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[21]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Mark S. Nixon,et al.  Using Gait as a Biometric, via Phase-weighted Magnitude Spectra , 1997, AVBPA.

[23]  Jiri Matas,et al.  On Combining Classifiers , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Trevor Darrell,et al.  On probabilistic combination of face and gait cues for identification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[25]  Azhin Tahir Sabir,et al.  Gait recognition based on Kinect sensor , 2014, Photonics Europe.

[26]  Fabio Tozeto Ramos,et al.  Unsupervised clustering of people from ‘skeleton’ data , 2012, 2012 7th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  David J. Hill,et al.  Deterministic Learning Theory , 2009 .

[28]  Tieniu Tan,et al.  Fusion of static and dynamic body biometrics for gait recognition , 2004, IEEE Trans. Circuits Syst. Video Technol..

[29]  Tardi Tjahjadi,et al.  Gait recognition based on shape and motion analysis of silhouette contours , 2013, Comput. Vis. Image Underst..

[30]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Tardi Tjahjadi,et al.  Silhouette-based gait recognition using Procrustes shape analysis and elliptic Fourier descriptors , 2012, Pattern Recognit..

[32]  Sudeep Sarkar,et al.  The gait identification challenge problem: data sets and baseline algorithm , 2002, Object recognition supported by user interaction for service robots.

[33]  Neha Jain,et al.  Gait recognition based on gait pal and pal entropy image , 2013, 2013 IEEE International Conference on Image Processing.

[34]  Manuel J. Marín-Jiménez,et al.  Multimodal features fusion for gait, gender and shoes recognition , 2016, Machine Vision and Applications.

[35]  Yasushi Makihara,et al.  The OU-ISIR Gait Database Comprising the Treadmill Dataset , 2012, IPSJ Trans. Comput. Vis. Appl..

[36]  Mark S. Nixon,et al.  Gait Recognition , 2014, Computer Vision, A Reference Guide.

[37]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[38]  Larry S. Davis,et al.  EigenGait: Motion-Based Recognition of People Using Image Self-Similarity , 2001, AVBPA.

[39]  Shaogang Gong,et al.  Gait recognition using Gait Entropy Image , 2009, ICDP.

[40]  James Nga-Kwok Liu,et al.  Gait flow image: A silhouette-based gait representation for human identification , 2011, Pattern Recognit..

[41]  Cong Wang,et al.  Deterministic Learning and Rapid Dynamical Pattern Recognition , 2007, IEEE Transactions on Neural Networks.

[42]  Neil M. Robertson,et al.  Dynamic Distance-Based Shape Features for Gait Recognition , 2014, Journal of Mathematical Imaging and Vision.

[43]  Tieniu Tan,et al.  Walker Recognition Without Gait Cycle Estimation , 2007, ICB.

[44]  Hamid Soltanian-Zadeh,et al.  Gait Recognition Using Wavelet Packet Silhouette Representation and Transductive Support Vector Machines , 2009, 2009 2nd International Congress on Image and Signal Processing.

[45]  Wei Zeng,et al.  Human gait recognition via deterministic learning , 2012, Neural Networks.

[46]  Tieniu Tan,et al.  Automatic gait recognition based on statistical shape analysis , 2003, IEEE Trans. Image Process..

[47]  Qiong Wu,et al.  A complete dynamic model of five-link bipedal walking , 2003, Proceedings of the 2003 American Control Conference, 2003..

[48]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[49]  Tieniu Tan,et al.  Uniprojective Features for Gait Recognition , 2007, ICB.

[50]  Björn W. Schuller,et al.  The TUM Gait from Audio, Image and Depth (GAID) database: Multimodal recognition of subjects and traits , 2014, J. Vis. Commun. Image Represent..

[51]  Yasushi Makihara,et al.  Individuality-preserving Silhouette Extraction for Gait Recognition , 2015, IPSJ Trans. Comput. Vis. Appl..

[52]  Chen Wang,et al.  Chrono-Gait Image: A Novel Temporal Template for Gait Recognition , 2010, ECCV.

[53]  Horst Bunke,et al.  Combination of Classifiers on the Decision Level for Face Recognition , 1996 .

[54]  Tieniu Tan,et al.  Efficient Night Gait Recognition Based on Template Matching , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[55]  James J. Little,et al.  Incremental Learning for Video-Based Gait Recognition With LBP Flow , 2013, IEEE Transactions on Cybernetics.