Vision-Based Analysis of Small Groups in Pedestrian Crowds

Building upon state-of-the-art algorithms for pedestrian detection and multi-object tracking, and inspired by sociological models of human collective behavior, we automatically detect small groups of individuals who are traveling together. These groups are discovered by bottom-up hierarchical clustering using a generalized, symmetric Hausdorff distance defined with respect to pairwise proximity and velocity. We validate our results quantitatively and qualitatively on videos of real-world pedestrian scenes. Where human-coded ground truth is available, we find substantial statistical agreement between our results and the human-perceived small group structure of the crowd. Results from our automated crowd analysis also reveal interesting patterns governing the shape of pedestrian groups. These discoveries complement current research in crowd dynamics, and may provide insights to improve evacuation planning and real-time situation awareness during public disturbances.

[1]  H. Chaiklin,et al.  The Myth of the Madding Crowd , 1992 .

[2]  A. Doucet,et al.  A Tutorial on Particle Filtering and Smoothing: Fifteen years later , 2008 .

[3]  John Soldera,et al.  Understanding people motion in video sequences using Voronoi diagrams , 2007, Pattern Analysis and Applications.

[4]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Luc Van Gool,et al.  You'll never walk alone: Modeling social behavior for multi-target tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[6]  A. Schadschneider Cellular Automaton Approach to Pedestrian Dynamics - Theory , 2001, cond-mat/0112117.

[7]  Paul A. Viola,et al.  Detecting Pedestrians Using Patterns of Motion and Appearance , 2005, International Journal of Computer Vision.

[8]  Jean-Marc Odobez,et al.  Detecting queues at vending machines: A statistical layered approach , 2008, 2008 19th International Conference on Pattern Recognition.

[9]  Qian Yu Multiple Target Tracking Using Spatio-Temporal Monte Carlo Markov Chain Data Association , 2007 .

[10]  Michel Bierlaire,et al.  Behavioral Priors for Detection and Tracking of Pedestrians in Video Sequences , 2006, International Journal of Computer Vision.

[11]  Clark McPhail,et al.  Using Film to Analyze Pedestrian Behavior , 1982 .

[12]  Matt Anderson,et al.  Constrained animation of flocks , 2003, SCA '03.

[13]  Gérard G. Medioni,et al.  Multiple-Target Tracking by Spatiotemporal Monte Carlo Markov Chain Data Association , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  A. Aveni The not-so-lonely crowd: Friendship groups in collective behavior. , 1977 .

[15]  W. Whyte City: Rediscovering the Center , 1988 .

[16]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[17]  Sergio A. Velastin,et al.  Crowd analysis: a survey , 2008, Machine Vision and Applications.

[18]  Adolf D. May,et al.  Traffic Flow Fundamentals , 1989 .

[19]  Antonio Albiol,et al.  VIDEO ANALYSIS USING CORNER MOTION STATISTICS , 2009 .

[20]  Ramakant Nevatia,et al.  Learning to associate: HybridBoosted multi-target tracker for crowded scene , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Ognjen Arandjelovic,et al.  Crowd Detection from Still Images , 2008, BMVC.

[22]  Marshall F. Tappen,et al.  Learning pedestrian dynamics from the real world , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[23]  A. Zander,et al.  Group dynamics, research and theory , 1955 .

[24]  Serge J. Belongie,et al.  Counting Crowded Moving Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[25]  Ko Nishino,et al.  Tracking with local spatio-temporal motion patterns in extremely crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Nuno Vasconcelos,et al.  Analysis of Crowded Scenes using Holistic Properties , 2009 .

[27]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[28]  N. R. Johnson Panic at “The Who Concert Stampede”: An Empirical Assessment , 1987 .

[29]  K. Mardia,et al.  Statistical Shape Analysis , 1998 .

[30]  Myron Flickner,et al.  Detection and tracking of shopping groups in stores , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[31]  Takahiro Okabe,et al.  Using individuality to track individuals: Clustering individual trajectories in crowds using local appearance and frequency trait , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[32]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[33]  Daniel Gatica-Perez,et al.  Automatic nonverbal analysis of social interaction in small groups: A review , 2009, Image Vis. Comput..

[34]  A.G.A. Perera,et al.  Detecting Semantic Group Activities Using Relational Clustering , 2008, 2008 IEEE Workshop on Motion and video Computing.

[35]  Daniel Thalmann,et al.  Crowd patches: populating large-scale virtual environments for real-time applications , 2009, I3D '09.

[36]  D. Helbing,et al.  The Walking Behaviour of Pedestrian Social Groups and Its Impact on Crowd Dynamics , 2010, PloS one.

[37]  Hai Tao,et al.  A Viewpoint Invariant Approach for Crowd Counting , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[38]  Michel Bierlaire,et al.  Specification, estimation and validation of a pedestrian walking behaviour model , 2007 .

[39]  Osama Masoud,et al.  Estimating pedestrian counts in groups , 2008, Comput. Vis. Image Underst..

[40]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[41]  Tony P. Pridmore,et al.  Using social effects to guide tracking in complex scenes , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[42]  Roberto Cipolla,et al.  Unsupervised Bayesian Detection of Independent Motion in Crowds , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[43]  A. G. Amitha Perera,et al.  Video Activity Recognition in the Real World , 2008, AAAI.

[44]  Robert B. Fisher,et al.  Modelling Crowd Scenes for Event Detection , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[45]  W. Eric L. Grimson,et al.  Learning Semantic Scene Models by Trajectory Analysis , 2006, ECCV.

[46]  Wolfram Burgard,et al.  Multi-model Hypothesis Group Tracking and Group Size Estimation , 2010, Int. J. Soc. Robotics.

[47]  Gérard G. Medioni,et al.  Multiple Target Tracking Using Spatio-Temporal Markov Chain Monte Carlo Data Association , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Xiaolin Hu,et al.  Modeling group structures in pedestrian crowd simulation , 2010, Simul. Model. Pract. Theory.

[49]  Mubarak Shah,et al.  Abnormal crowd behavior detection using social force model , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  D. Campbell Common fate, similarity, and other indices of the status of aggregates of persons as social entities , 2007 .

[51]  Shaogang Gong,et al.  Recognition of group activities using dynamic probabilistic networks , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[52]  A. Marana,et al.  On the efficacy of texture analysis for crowd monitoring , 1998, Proceedings SIBGRAPI'98. International Symposium on Computer Graphics, Image Processing, and Vision (Cat. No.98EX237).

[53]  J.K. Aggarwal,et al.  Recognition of High-level Group Activities Based on Activities of Individual Members , 2008, 2008 IEEE Workshop on Motion and video Computing.

[54]  Anuj Srivastava,et al.  Statistical Shape Analysis , 2014, Computer Vision, A Reference Guide.

[55]  Dimitris N. Metaxas,et al.  Eurographics/ Acm Siggraph Symposium on Computer Animation (2007) Group Behavior from Video: a Data-driven Approach to Crowd Simulation , 2022 .

[56]  Soraia Raupp Musse,et al.  A Model of Human Crowd Behavior : Group Inter-Relationship and Collision Detection Analysis , 1997, Computer Animation and Simulation.

[57]  Anil M. Cheriyadat,et al.  Detecting Dominant Motions in Dense Crowds , 2008, IEEE Journal of Selected Topics in Signal Processing.

[58]  Dorwin Cartwright,et al.  Group dynamics: Research and theory, 2nd ed. , 1960 .

[59]  Mubarak Shah,et al.  Floor Fields for Tracking in High Density Crowd Scenes , 2008, ECCV.

[60]  Ramakant Nevatia,et al.  Bayesian human segmentation in crowded situations , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[61]  Robert T. Collins,et al.  Marked point processes for crowd counting , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[62]  Nuno Vasconcelos,et al.  Privacy preserving crowd monitoring: Counting people without people models or tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Victor J. Blue,et al.  Cellular automata microsimulation for modeling bi-directional pedestrian walkways , 2001 .

[64]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[65]  Monique Thonnat,et al.  Tracking Groups of People for Video Surveillance , 2002 .

[66]  Takeo Kanade,et al.  Tracking in unstructured crowded scenes , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[67]  Feng Chen,et al.  Hierarchical group process representation in multi-agent activity recognition , 2008, Signal Process. Image Commun..

[68]  William M. Rand,et al.  Objective Criteria for the Evaluation of Clustering Methods , 1971 .

[69]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[70]  Edward T. Hall,et al.  A System for the Notation of Proxemic Behavior1 , 1963 .

[71]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).