Groups and Crowds: Behaviour Analysis of People Aggregations

Automatic analysis of human behavior in social environment is a key topic for the computer vision community, with applications in security and video surveillance. While human behavior at an individual (single person) level has been widely studied in the past years, analysis of groups and crowd behavior, is still at a preliminary stage, with room for new approaches to emerge. Recently, there has been significant research effort dedicated to the development of automated computer vision techniques, intended to enhance safety of our societies by monitoring human behaviors and their actions in groups and crowd level. In particular, groups are usually formed by number of people who gathered for private meeting, birthday party, or wedding, while we consider crowd as huge number of people are gathered together to participate for a national or religious event, or protest due to some dissatisfaction. In this chapter, we will provide a broad overview on proposed approaches on human behavior analysis in group and crowd level, as well as, a detailed of some most recent state-of-the-art methods along with extensive experiments and comparison.

[1]  G. Batchelor,et al.  An Introduction to Fluid Dynamics , 1968 .

[2]  Vittorio Murino,et al.  Characterizing Humans on Riemannian Manifolds , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Alessio Del Bue,et al.  Social interaction discovery by statistical analysis of F-formations , 2011, BMVC.

[4]  Sergio A. Velastin,et al.  Crowd analysis: a survey , 2008, Machine Vision and Applications.

[5]  Ramin Mehran,et al.  Abnormal crowd behavior detection using social force model , 2009, CVPR.

[6]  T. M. Ciolek,et al.  Environment and the Spatial Arrangement of Conversational Encounters , 1980 .

[7]  Tal Hassner,et al.  Violent flows: Real-time detection of violent crowd behavior , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[8]  Mubarak Shah,et al.  Person-on-person violence detection in video data , 2002, Object recognition supported by user interaction for service robots.

[9]  Dirk Helbing,et al.  How simple rules determine pedestrian behavior and crowd disasters , 2011, Proceedings of the National Academy of Sciences.

[10]  Francesco Setti,et al.  Group detection in still images by F-formation modeling: A comparative study , 2013, 2013 14th International Workshop on Image Analysis for Multimedia Interactive Services (WIAMIS).

[11]  Ben J. A. Kröse,et al.  Detecting F-formations as dominant sets , 2011, ICMI '11.

[12]  Leo Grady,et al.  A multilevel banded graph cuts method for fast image segmentation , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[13]  Alessandro Perina,et al.  Crowd motion monitoring using tracklet-based commotion measure , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[14]  Shaogang Gong,et al.  Security and Surveillance , 2011, Visual Analysis of Humans.

[15]  Bingbing Ni,et al.  Crowded Scene Analysis: A Survey , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Francesco Setti,et al.  F-Formation Detection: Individuating Free-Standing Conversational Groups in Images , 2015, PloS one.

[17]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[18]  Nuno Vasconcelos,et al.  Privacy preserving crowd monitoring: Counting people without people models or tracking , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[20]  Yangsheng Xu,et al.  Abnormal crowd motion analysis , 2009, 2009 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[21]  Jean-Marc Odobez,et al.  Tracking the Visual Focus of Attention for a Varying Number of Wandering People , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Francesco Setti,et al.  Multi-scale f-formation discovery for group detection , 2013, 2013 IEEE International Conference on Image Processing.

[23]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[24]  Roberto Brunelli,et al.  Joint Bayesian Tracking of Head Location and Pose from Low-Resolution Video , 2007, CLEAR.

[25]  Serge J. Belongie,et al.  Counting Crowded Moving Objects , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[26]  Alessandro Perina,et al.  Angry Crowds: Detecting Violent Events in Videos , 2016, ECCV.

[27]  Peter H. Tu,et al.  Simultaneous estimation of segmentation and shape , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[28]  T. M. Ciolek.  The proxemics lexicon: A first approximation , 1983 .

[29]  Alessandro Perina,et al.  A comparison of crowd commotion measures from generative models , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Pushmeet Kohli,et al.  Inference Methods for CRFs with Co-occurrence Statistics , 2012, International Journal of Computer Vision.

[31]  Mubarak Shah,et al.  A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Qiang Wu,et al.  Violent video detection based on MoSIFT feature and sparse coding , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[33]  Marcello Pelillo,et al.  Detecting conversational groups in images and sequences: A robust game-theoretic approach , 2016, Comput. Vis. Image Underst..

[34]  Nebojsa Jojic,et al.  Multidimensional counting grids: Inferring word order from disordered bags of words , 2011, UAI.

[35]  Bernt Schiele,et al.  Learning People Detectors for Tracking in Crowded Scenes , 2013, 2013 IEEE International Conference on Computer Vision.

[36]  Mario Vento,et al.  A Method for Counting People in Crowded Scenes , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[37]  Takeo Kanade,et al.  Tracking in unstructured crowded scenes , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[38]  Ning Xu,et al.  Object segmentation using graph cuts based active contours , 2007, Comput. Vis. Image Underst..

[39]  Marcello Pelillo,et al.  A Game-Theoretic Probabilistic Approach for Detecting Conversational Groups , 2014, ACCV.

[40]  Robert B. Fisher,et al.  The BEHAVE video dataset: ground truthed video for multi-person behavior classification , 2010 .

[41]  Jean-Marc Odobez,et al.  We are not contortionists: Coupled adaptive learning for head and body orientation estimation in surveillance video , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[42]  Mubarak Shah,et al.  Chaotic invariants of Lagrangian particle trajectories for anomaly detection in crowded scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  Alessandro Perina,et al.  Violence detection in crowded scenes using substantial derivative , 2015, 2015 12th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[44]  Vittorio Murino,et al.  Social interactions by visual focus of attention in a three‐dimensional environment , 2013, Expert Syst. J. Knowl. Eng..

[45]  Aggelos K. Katsaggelos,et al.  Detecting contextual anomalies of crowd motion in surveillance video , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[46]  Louis Kratz,et al.  Anomaly detection in extremely crowded scenes using spatio-temporal motion pattern models , 2009, CVPR.

[47]  A. Kendon.  Conducting Interaction: Patterns of Behavior in Focused Encounters , 1990 .

[48]  Alessandro Perina,et al.  Analyzing Tracklets for the Detection of Abnormal Crowd Behavior , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[49]  Alessandro Perina,et al.  Detecting Abnormal Behavioral Patterns in Crowd Scenarios , 2016, Toward Robotic Socially Believable Behaving Systems.

[50]  Haidi Ibrahim,et al.  Recent survey on crowd density estimation and counting for visual surveillance , 2015, Eng. Appl. Artif. Intell..

[51]  Jonathan D. Nelson,et al.  Simple Heuristics and the Modelling of Crowd Behaviours , 2014 .

[52]  Marcello Pelillo,et al.  Dominant Sets and Pairwise Clustering , 2007 .

[53]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[54]  Ioannis A. Kakadiaris,et al.  Social Cues in Group Formation and Local Interactions for Collective Activity Analysis , 2013, VISAPP.

[55]  Mubarak Shah,et al.  Floor Fields for Tracking in High Density Crowd Scenes , 2008, ECCV.

[56]  H. Bozdogan.  Model selection and Akaike's Information Criterion (AIC): The general theory and its analytical extensions , 1987 .

[57]  Roberto Cipolla,et al.  Automatic 3D object segmentation in multiple views using volumetric graph-cuts , 2007, Image Vis. Comput..

[58]  Oswald Lanz,et al.  Approximate Bayesian multibody tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[59]  Chee Seng Chan,et al.  Crowd behavior analysis: A review where physics meets biology , 2015, Neurocomputing.

[60]  M. Cook.  Experiments on Orientation and Proxemics , 1970 .

[61]  Jean-Marc Odobez,et al.  Multiperson Visual Focus of Attention from Head Pose and Meeting Contextual Cues , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[62]  Xiaogang Wang,et al.  Scene-Independent Group Profiling in Crowd , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Xiaofeng Li,et al.  Abnormal crowd behavior detection based on optical flow and dynamic threshold , 2014, Proceeding of the 11th World Congress on Intelligent Control and Automation.

[64]  Helbing,et al.  Social force model for pedestrian dynamics. , 1995, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.