On a three-dimensional gait recognition system

The University of Southampton Multi-Biometric Tunnel is a high performance data-capture and recognition system; designed with airports and other busy public areas in mind. It is able to acquire a variety of non-contact biometrics in a non-intrusive manner, requiring minimal subject cooperation. The system uses twelve cameras to record gait and perform three-dimensional reconstruction; the use of volumetric data avoids the problems caused by viewpoint dependence - a serious problem for many gait analysis approaches. The early prototype by Middleton et al. was used as the basis for creating a new and improved system, designed for the collection of a new large dataset, containing gait, face and ear. Extensive modifications were made, including new software for managing the data collection experiment and processing the dataset. Rigorous procedures were implemented to protect the privacy of participants and ensure consistency between capture sessions. Collection of the new multi-biometric dataset spanned almost one year; resulting in over 200 subjects and 2000 samples. Experiments performed on the newly collected dataset resulted in excellent recognition performance, with all samples correctly classified and a 1.58% equal error rate; the matching of subjects against previous samples was also found to be reasonably accurate. The fusion of gait with a simple facial analysis technique found the addition of gait to be beneficial -- especially at a distance. Further experiments investigated the effect of static and dynamic features, camera misalignment, average silhouette resolution, camera layout, and the matching of outdoor video footage against data from the Biometric Tunnel. The results in this thesis prove significant due to the unprecedented size of the new dataset and the excellent recognition performance achieved; providing a significant body of evidence to support the argument that an individual's gait is unique. L. Middleton, D. K. Wagg, A. I. Bazin, J. N. Carter and M. S. Nixon. A smart environment for biometric capture. Automation Science and Engineering, Proceedings of IEEE International Conference on, 57-62, 2006.

[1]  Carlos Orrite-Uruñuela,et al.  2D silhouette and 3D skeletal models for human detection and tracking , 2004, ICPR 2004.

[2]  A. N. Rajagopalan,et al.  Gait-based recognition of humans using continuous HMMs , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[3]  Mark S. Nixon,et al.  On automated model-based extraction and analysis of gait , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[4]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[5]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[6]  Takeo Kanade,et al.  A real time system for robust 3D voxel reconstruction of human motions , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[7]  Mark S. Nixon,et al.  Is Enough Enough? What Is Sufficiency in Biometric Data? , 2006, ICIAR.

[8]  Mark S. Nixon,et al.  What image information is important in silhouette-based gait recognition? , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[9]  Hua Li,et al.  Amplitude spectrum-based gait recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[10]  Ron Kimmel,et al.  Demosaicing: Image Reconstruction from Color CCD Samples , 1998, ECCV.

[11]  Mark S. Nixon,et al.  Gait Extraction and Description by Evidence-Gathering , 1999 .

[12]  Li Liu,et al.  Kernel-Based Method for Automated Walking Patterns Recognition Using Kinematics Data , 2006, ICNC.

[13]  Adam Prügel-Bennett,et al.  Automatic gait recognition using area-based metrics , 2003 .

[14]  Lily Lee,et al.  Gait analysis for classification , 2002 .

[15]  Mark S. Nixon,et al.  Probabilistic combination of static and dynamic gait features for verification , 2005, SPIE Defense + Commercial Sensing.

[16]  Patrick J. Flynn,et al.  Multi-Modal 2D and 3D Biometrics for Face Recognition , 2003, AMFG.

[17]  Nikolaos V. Boulgouris,et al.  Gait Representation and Recognition Based on Radon Transform , 2006, 2006 International Conference on Image Processing.

[18]  Larry S. Davis,et al.  EigenGait: Motion-Based Recognition of People Using Image Self-Similarity , 2001, AVBPA.

[19]  Chiraz Ben Abdelkader.  Motion-Based Recognition of People in EigenGait Space , 2002 .

[20]  A. Laurentini,et al.  The Visual Hull Concept for Silhouette-Based Image Understanding , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Mark S. Nixon,et al.  Recognising human and animal movement by symmetry , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[22]  C. Dyer.  Volumetric Scene Reconstruction from Multiple Views , 2001 .

[23]  Mark S. Nixon,et al.  Automated person recognition by walking and running via model-based approaches , 2004, Pattern Recognit..

[24]  Neil A. Thacker,et al.  Performance characterization in computer vision: A guide to best practices , 2008, Comput. Vis. Image Underst..

[25]  Mark S. Nixon,et al.  On Acquisition and Analysis of a Dataset Comprising of Gait, Ear and Semantic data , 2011 .

[26]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[27]  G. Johansson.  Visual perception of biological motion and a model for its analysis , 1973 .

[28]  Gang Xu,et al.  Understanding human motion patterns , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[29]  G. Higginson,et al.  The kinematics of hip joints: normal functioning. , 1984, Clinical physics and physiological measurement : an official journal of the Hospital Physicists' Association, Deutsche Gesellschaft fur Medizinische Physik and the European Federation of Organisations for Medical Physics.

[30]  John N. Carter,et al.  Spatio-temporal 3D Gait Recognition , 2008 .

[31]  Tieniu Tan,et al.  Gait recognition based on Procrustes shape analysis , 2002, Proceedings. International Conference on Image Processing.

[32]  Edward Y. Chang,et al.  Color filter array recovery using a threshold-based variable number of gradients , 1999, Electronic Imaging.

[33]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[34]  Giulio Sandini,et al.  3D object reconstruction using stereo and motion , 1989, IEEE Trans. Syst. Man Cybern..

[35]  T. Poggio,et al.  A parallel algorithm for real-time computation of optical flow , 1989, Nature.

[36]  Tieniu Tan,et al.  Automatic gait recognition based on statistical shape analysis , 2003, IEEE Trans. Image Process..

[37]  M. Nixon,et al.  Canonical space representation for recognizing humans by gait and face , 1998, 1998 IEEE Southwest Symposium on Image Analysis and Interpretation (Cat. No.98EX165).

[38]  Mark S. Nixon,et al.  Extended Model-Based Automatic Gait Recognition of Walking and Running , 2001, AVBPA.

[39]  Stephanie R. Taylor,et al.  Analysis and recognition of walking movements , 2002, Object recognition supported by user interaction for service robots.

[40]  Tom Drummond,et al.  ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition , 2009, BMVC.

[41]  Richard Szeliski,et al.  Recovering 3D shape and motion from image streams using nonlinear least squares , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Larry S. Davis,et al.  Stride and cadence as a biometric in automatic person identification and verification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[43]  Ching-Han Chen,et al.  Fusion of Face and Iris Features for Multimodal Biometrics , 2006, ICB.

[44]  M. Lee,et al.  The University of Southampton Multi-Biometric Tunnel and introducing a novel 3D gait dataset , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.

[45]  Arun Ross,et al.  An introduction to biometric recognition , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[46]  Mark S. Nixon,et al.  Automatic Recognition by Gait , 2006, Proceedings of the IEEE.

[47]  Yanxi Liu,et al.  Gait Sequence Analysis Using Frieze Patterns , 2002, ECCV.

[48]  Mark S. Nixon,et al.  Zernike Velocity Moments for Description and Recognition of Moving Shapes , 2001, BMVC.

[49]  Aaron F. Bobick,et al.  Gait recognition from time-normalized joint-angle trajectories in the walking plane , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[50]  Sylvia C. Wong,et al.  Lightweight Agent Framework for Camera Array Applications , 2005, KES.

[51]  Qiang He,et al.  Individual recognition from periodic activity using hidden Markov models , 2000, Proceedings Workshop on Human Motion.

[52]  Timothy F. Cootes,et al.  Training Models of Shape from Sets of Examples , 1992, BMVC.

[53]  Yap-Peng Tan,et al.  View invariant gait recognition , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[54]  Mark S. Nixon,et al.  Zernike velocity moments for sequence-based description of moving features , 2006, Image Vis. Comput..

[55]  Mark S. Nixon,et al.  Markerless view independent gait analysis with self-camera calibration , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[56]  Josef Kittler,et al.  Sum Versus Vote Fusion in Multiple Classifier Systems , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[57]  Richard O. Duda,et al.  Use of the Hough transformation to detect lines and curves in pictures , 1972, CACM.

[58]  Svetha Venkatesh,et al.  Temporal PDMs for gait classification , 2002, Object recognition supported by user interaction for service robots.

[59]  A. B. Drought,et al.  WALKING PATTERNS OF NORMAL MEN. , 1964, The Journal of bone and joint surgery. American volume.

[60]  Larry S. Davis,et al.  A Robust Background Subtraction and Shadow Detection , 1999 .

[61]  Arun Ross,et al.  Multibiometric systems , 2004, CACM.

[62]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[63]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[64]  Hua Li,et al.  3D gait recognition using multiple cameras , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[65]  M. Nixon,et al.  Statistical gait recognition via velocity moments , 2000 .

[66]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[67]  G. Mather,et al.  Gender discrimination in biological motion displays based on dynamic cues , 1994, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[68]  Ping-Sing Tsai,et al.  Shape from Shading: A Survey , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[69]  Sudeep Sarkar,et al.  The gait identification challenge problem: data sets and baseline algorithm , 2002, Object recognition supported by user interaction for service robots.

[70]  Shaogang Gong,et al.  Fusing gait and face cues for human gender recognition , 2008, Neurocomputing.

[71]  Arun Ross,et al.  Multimodal biometrics: An overview , 2004, 2004 12th European Signal Processing Conference.

[72]  Darius Burschka,et al.  Advances in Computational Stereo , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  Mark S. Nixon,et al.  Human Perambulation as a Self Calibrating Biometric , 2007, AMFG.

[74]  Jason M. Nash,et al.  Automatic gait recognition , 1999 .

[75]  Aaron F. Bobick,et al.  Gait recognition using static, activity-specific parameters , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[76]  Tieniu Tan,et al.  Fusion of static and dynamic body biometrics for gait recognition , 2003, IEEE Transactions on Circuits and Systems for Video Technology.

[77]  J. Little,et al.  Describing motion for recognition , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[78]  Mark S. Nixon,et al.  On a Large Sequence-Based Human Gait Database , 2004 .

[79]  Robert T. Collins,et al.  Gait Shape Estimation for Identification , 2003, AVBPA.

[80]  Tieniu Tan,et al.  Human identification based on gait , 2005, The Kluwer international series on biometrics.

[81]  Z. Liu,et al.  Simplest representation yet for gait recognition: averaged silhouette , 2004, ICPR 2004.

[82]  Takumi Kobayashi,et al.  Action and simultaneous multiple-person identification using cubic higher-order local auto-correlation , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[83]  Patrick J. Flynn,et al.  A survey of approaches to three-dimensional face recognition , 2004, ICPR 2004.

[84]  David Zhang,et al.  Human gait recognition by the fusion of motion and static spatio-temporal templates , 2007, Pattern Recognit..

[85]  Alex Pentland,et al.  Face recognition using eigenfaces , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[86]  Larry S. Davis,et al.  Foundations of Image Understanding , 2001 .

[87]  Trevor Darrell,et al.  Integrated face and gait recognition from multiple views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[88]  Helen C. Shen,et al.  Personal Verification Using Palmprint and Hand Geometry Biometric , 2003, AVBPA.

[89]  W. Eric L. Grimson,et al.  Gait analysis for recognition and classification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[90]  Pascal Fua,et al.  3D tracking for gait characterization and recognition , 2004, Sixth IEEE International Conference on Automatic Face and Gesture Recognition, 2004. Proceedings..

[91]  J. Little,et al.  Recognizing People by Their Gait: The Shape of Motion , 1998 .

[92]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[93]  Bir Bhanu,et al.  Statistical feature fusion for gait-based human recognition , 2004, CVPR 2004.

[94]  Thomas W. Parks,et al.  RECONSTRUCTION OF COLOR IMAGES FROM CCD ARRAYS , 2000 .

[95]  Neil A. Thacker,et al.  Performance characterisation in computer vision: statistics in testing and design , 2001 .

[96]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[97]  Thomas Malzbender,et al.  A Survey of Methods for Volumetric Scene Reconstruction from Photographs , 2001, VG.

[98]  Imed Bouchrika,et al.  Gait analysis and recognition for automated visual surveillance , 2008 .

[99]  Trevor Darrell,et al.  Background estimation and removal based on range and color , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[100]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[101]  Jack Bresenham,et al.  Algorithm for computer control of a digital plotter , 1965, IBM Syst. J..

[102]  Olaf Munkelt,et al.  Adaptive Background Estimation and Foreground Detection using Kalman-Filtering , 1995 .

[103]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[104]  Mark S. Nixon,et al.  Using Gait as a Biometric, via Phase-weighted Magnitude Spectra , 1997, AVBPA.

[105]  Mark S. Nixon,et al.  Developing a non-intrusive biometric environment , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[106]  R. Plankers,et al.  Articulated soft objects for video-based body modeling , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[107]  John N. Carter,et al.  Towards pose invariant gait reconstruction , 2005, IEEE International Conference on Image Processing 2005.

[108]  Tieniu Tan,et al.  A new attempt to gait-based human identification , 2002, Object recognition supported by user interaction for service robots.

[109]  Thomas W. Parks,et al.  Adaptive homogeneity-directed demosaicing algorithm , 2005, IEEE Transactions on Image Processing.

[110]  Mark S. Nixon,et al.  Automatic extraction and description of human gait models for recognition purposes , 2003, Comput. Vis. Image Underst..

[111]  Mark S. Nixon,et al.  Statistical gait description via temporal moments , 2000, 4th IEEE Southwest Symposium on Image Analysis and Interpretation.

[112]  Bir Bhanu,et al.  Human Recognition on Combining Kinematic and Stationary Features , 2003, AVBPA.

[113]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[114]  Hiroshi Murase,et al.  Moving object recognition in eigenspace representation: gait analysis and lip reading , 1996, Pattern Recognit. Lett..

[115]  Mark S. Nixon,et al.  Model-Based Feature Extraction for Gait Analysis and Recognition , 2007, MIRAGE.

[116]  Rama Chellappa,et al.  A hidden Markov model based framework for recognition of humans from gait sequences , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[117]  Jake K. Aggarwal,et al.  TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE , 2008 .

[118]  John N. Carter,et al.  Automatic recognition by gait: progress and prospects , 2003 .

[119]  Robert T. Collins,et al.  Silhouette-based human identification from body shape and gait , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[120]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[121]  Jeffrey E. Boyd.  Video Phase-Locked Loops in Gait Recognition , 2001, ICCV.

[122]  Timothy F. Cootes,et al.  Active Appearance Models , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[123]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[124]  John Daugman,et al.  Biometric decision landscapes , 2000 .

[125]  M.S. Nixon,et al.  The Use of Semantic Human Description as a Soft Biometric , 2008, 2008 IEEE Second International Conference on Biometrics: Theory, Applications and Systems.