Applications of a Simple Characterization of Human Gait in Surveillance

Applications of a simple spatiotemporal characterization of human gait in the surveillance domain are presented. The approach is based on decomposing a video sequence into x-t slices, which generate periodic patterns referred to as double helical signatures (DHSs). The features of DHS are given as follows: 1) they naturally encode the appearance and kinematics of human motion and reveal geometric symmetries and 2) they are effective and efficient for recovering gait parameters and detecting simple events. We present an iterative local curve embedding algorithm to extract the DHS from video sequences. Two applications are then considered. First, the DHS is used for simultaneous segmentation and labeling of body parts in cluttered scenes. Experimental results showed that the algorithm is robust to size, viewing angles, camera motion, and severe occlusion. Then, the DHS is used to classify load-carrying conditions. By examining various symmetries in DHS, activities such as carrying, holding, and walking with objects that are attached to legs are detected. Our approach possesses several advantages: a compact representation that can be computed in real time is used; furthermore, it does not depend on silhouettes or landmark tracking, which are sensitive to errors in background subtraction stage.

[1]  Michael J. Black,et al.  Parameterized Modeling and Recognition of Activities , 1999, Comput. Vis. Image Underst..

[2]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Lihi Zelnik-Manor,et al.  Event-based analysis of video , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[4]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[5]  David Wright,et al.  Indra's Pearls: The Vision of Felix Klein , 2002 .

[6]  Matti Pietikäinen,et al.  Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Ben J. A. Kröse,et al.  A k-segments algorithm for finding principal curves , 2002, Pattern Recognit. Lett..

[8]  E. Adelson,et al.  Analyzing gait with spatiotemporal surfaces , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[9]  Jitendra Malik,et al.  Recognizing action at a distance , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[10]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[11]  Mubarak Shah,et al.  Cyclic motion detection for motion based recognition , 1994, Pattern Recognit..

[12]  Fang Liu,et al.  Finding periodicity in space and time , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[13]  Sudeep Sarkar,et al.  Effect of silhouette quality on hard problems in gait recognition , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[14]  Yaser Sheikh,et al.  Exploring the space of a human action , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[15]  Larry S. Davis,et al.  Backpack: Detection of People Carrying Objects Using Silhouettes , 2001, Comput. Vis. Image Underst..

[16]  M. Allmen.  Image sequence description using spatiotemporal flow curves: toward motion-based recognition , 1992 .

[17]  Mubarak Shah,et al.  Motion-Based Recognition , 1997, Computational Imaging and Vision.

[18]  Rama Chellappa,et al.  Finding Gait in Space and Time , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[19]  Dariu Gavrila,et al.  The Visual Analysis of Human Movement: A Survey , 1999, Comput. Vis. Image Underst..

[20]  Eli Shechtman,et al.  Space-time behavior based correlation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[21]  T. Hastie,et al.  Principal Curves , 2007 .

[22]  Mark S. Nixon,et al.  Human identification by spatio-temporal symmetry , 2002, Object recognition supported by user interaction for service robots.

[23]  Robert C. Bolles,et al.  Epipolar-plane image analysis: An approach to determining structure from motion , 1987, International Journal of Computer Vision.

[24]  Yanxi Liu,et al.  A computational model for periodic pattern perception based on frieze and wallpaper groups , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  Yanxi Liu,et al.  Gait Sequence Analysis Using Frieze Patterns , 2002, ECCV.

[27]  Yanxi Liu,et al.  Shape Variation-Based Frieze Pattern for Robust Gait Recognition , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Dimitris N. Metaxas,et al.  Automating gait generation , 2001, SIGGRAPH.

[29]  Riad I. Hammoud,et al.  Thermal-Visible Video Fusion for Moving Target Tracking and Pedestrian Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  M. Nixon,et al.  Extraction and recognition of periodically deforming objects by continuous, spatio-temporal shape description , 2004, CVPR 2004.

[31]  Steven M. Seitz,et al.  View-Invariant Analysis of Cyclic Motion , 1997, International Journal of Computer Vision.

[32]  Robert C. Bolles,et al.  Generalizing Epipolar-Plane Image Analysis on the spatiotemporal surface , 2004, International Journal of Computer Vision.

[33]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[34]  Stefano Soatto,et al.  Dynamic Textures , 2003, International Journal of Computer Vision.

[35]  Sudeep Sarkar,et al.  The gait identification challenge problem: data sets and baseline algorithm , 2002, Object recognition supported by user interaction for service robots.

[36]  Xuelong Li,et al.  Human Carrying Status in Visual Surveillance , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[37]  Mark S. Nixon,et al.  Automatic Recognition by Gait , 2006, Proceedings of the IEEE.

[38]  José M. F. Moura,et al.  Capture and Representation of Human Walking in Live Video Sequences , 1999, IEEE Trans. Multim..

[39]  L. R. Rabiner,et al.  A comparative study of several dynamic time-warping algorithms for connected-word recognition , 1981, The Bell System Technical Journal.

[40]  Ahmed M. Elgammal,et al.  Carrying Object Detection Using Pose Preserving Dynamic Shape Models , 2006, AMDO.

[41]  Ramesh C. Jain,et al.  Qualitative motion analysis using a spatio-temporal approach , 1991, Proceedings. 1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  Jeffrey E. Boyd,et al.  Synchronization of oscillations for machine perception of gaits , 2004, Comput. Vis. Image Underst..

[43]  J. Little,et al.  Recognizing People by Their Gait: The Shape of Motion , 1998 .

[44]  V. M. Zat︠s︡iorskiĭ.  Kinematics of human motion , 1998 .

[45]  Sudeep Sarkar,et al.  Perceptual Organization Based Computational Model for Robust Segmentation of Moving Objects , 2002, Comput. Vis. Image Underst..

[46]  Randal C. Nelson,et al.  Detection and Recognition of Periodic, Nonrigid Motion , 1997, International Journal of Computer Vision.

[47]  Jiebo Luo,et al.  Body Localization in Still Images Using Hierarchical Models and Hybrid Search , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[48]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.