Multi-view multi-modal person authentication from a single walking image sequence

This paper describes a method for multi-view multimodal biometrics from a single walking image sequence. As multi-modal cues, we adopt not only face and gait but also the actual height of a person, all of which are simultaneously captured by a single camera. As multi-view cues, we use the variation in the observation views included in a single image sequence captured by a camera with a relatively wide field of view. This enables us to improve the authentication of a person based on multiple modalities and views, while retaining the potential for real applications such as surveillance and forensics using only a single image sequence (a single session with a single camera). In the experiments, we constructed a large-scale multi-view multimodal score data set with 1,912 subjects, and evaluated the proposed method using the data set in a statistically reliable way. We achieved 0.37% equal error rates for the false acceptance and rejection rates in the verification scenarios, and 99.15% rank-1 identification rate in the identification scenarios.

[1]  Yasushi Makihara,et al.  Silhouette transformation based on walking speed for gait identification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[2]  Xiaoli Zhou,et al.  Human Recognition at a Distance in Video by Integrating Face Profile and Gait , 2005, AVBPA.

[3]  Xiaoli Zhou,et al.  Integrating Face and Gait for Human Recognition at a Distance in Video , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[4]  Michael Mayo,et al.  3D Face Recognition Using Multiview Keypoint Matching , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[5]  Xiaoli Zhou,et al.  Feature fusion of side face and gait for video-based human identification , 2008, Pattern Recognit..

[6]  Amit K. Roy-Chowdhury,et al.  A study on view-insensitive gait recognition , 2005, IEEE International Conference on Image Processing 2005.

[7]  Kate Smith-Miles,et al.  Context-aware fusion: A case study on fusion of gait and face for human identification in video , 2010, Pattern Recognit..

[8]  Trevor Darrell,et al.  Integrated face and gait recognition from multiple views , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[9]  Rama Chellappa,et al.  Fusion of gait and face for human identification , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[10]  Surendra Ranganath,et al.  Fusion of Chaotic Measure Into a New Hybrid Face-Gait System for Human Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[11]  Julian Fiérrez,et al.  Target dependent score normalization techniques and their application to signature verification , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[12]  Chris Boehnen,et al.  A multi-sample standoff multimodal biometric system , 2012, 2012 IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS).

[13]  Anil K. Jain,et al.  A Principled Approach to Score Level Fusion in Multimodal Biometric Systems , 2005, AVBPA.

[14]  Kate Smith-Miles,et al.  Adaptive Fusion of Gait and Face for Human Identification in Video , 2008, 2008 IEEE Workshop on Applications of Computer Vision.

[15]  Yasushi Makihara,et al.  Gait Recognition Using a View Transformation Model in the Frequency Domain , 2006, ECCV.

[16]  Andrea F. Abate,et al.  2D and 3D face recognition: A survey , 2007, Pattern Recognit. Lett..

[17]  Takeo Kanade,et al.  Multi-PIE , 2008, 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition.

[18]  Trevor Darrell,et al.  On probabilistic combination of face and gait cues for identification , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[19]  Tieniu Tan,et al.  Human identification based on gait , 2005, The Kluwer international series on biometrics.

[20]  Josef Kittler,et al.  Discriminative Learning and Recognition of Image Set Classes Using Canonical Correlations , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Marie-Pierre Jolly,et al.  Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images , 2001, ICCV.

[22]  Qiang Wu,et al.  Support vector regression for multi-view gait recognition based on local motion feature selection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[23]  Masashi Nishiyama,et al.  Face Recognition with the Multiple Constrained Mutual Subspace Method , 2003, AVBPA.

[24]  Gerhard Rigoll,et al.  Combined face and gait recognition using alpha matte preprocessing , 2012, 2012 5th IAPR International Conference on Biometrics (ICB).

[25]  Alexander G. Hauptmann,et al.  Towards robust face recognition from multiple views , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[26]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Rama Chellappa,et al.  Combining multiple evidences for gait recognition , 2003, 2003 IEEE International Conference on Acoustics, Speech, and Signal Processing, 2003. Proceedings. (ICASSP '03)..

[28]  Rabia Jafri,et al.  A Survey of Face Recognition Techniques , 2009, J. Inf. Process. Syst..

[29]  Julian Fierrez,et al.  Dealing with sensor interoperability in multi-biometrics: the UPM experience at the Biosecure Multimodal Evaluation 2007 , 2008, SPIE Defense + Commercial Sensing.

[30]  Xuelong Li,et al.  Multimodal biometrics using geometry preserving projections , 2008, Pattern Recognit..

[31]  Rama Chellappa,et al.  Towards a view invariant gait recognition algorithm , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[32]  Sudeep Sarkar,et al.  Outdoor recognition at a distance by fusing gait and face , 2007, Image Vis. Comput..

[33]  Yasushi Makihara,et al.  The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[34]  Marie-Pierre Jolly,et al.  Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[35]  Yasushi Yagi,et al.  HyperOmni vision: Visual navigation with an omnidirectional image sensor , 1997 .

[36]  Yasushi Makihara,et al.  Clothing-invariant gait identification using part-based clothing categorization and adaptive weight control , 2010, Pattern Recognit..

[37]  Osamu Yamaguchi,et al.  Face Recognition Using Multi-viewpoint Patterns for Robot Vision , 2003, ISRR.

[38]  Xiaoli Zhou,et al.  Feature Fusion of Face and Gait for Human Recognition at a Distance in Video , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[39]  Arun Ross,et al.  Handbook of Multibiometrics , 2006, The Kluwer international series on biometrics.

[40]  Azriel Rosenfeld,et al.  Face recognition: A literature survey , 2003, CSUR.

[41]  Yasushi Makihara,et al.  Which Reference View is Effective for Gait Identification Using a View Transformation Model? , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[42]  Qiang Wu,et al.  Gait Recognition Under Various Viewing Angles Based on Correlated Motion Regression , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[43]  Tieniu Tan,et al.  Gait Recognition Based on Fusion of Multi-view Gait Sequences , 2006, ICB.

[44]  Yasushi Makihara,et al.  Gait Identification Based on Multi-view Observations Using Omnidirectional Camera , 2007, ACCV.