Comparison of Main Approaches for Extracting Behavior Features from Crowd Flow Analysis

Extracting features from crowd flow analysis has become an important research challenge due to its social cost and the impact of inadequate planning of high-quality services and security monitoring on the lives of citizens. This paper descriptively reviews and compares existing crowd analysis approaches based on different data sources. This survey provides the fundamentals of crowd analysis and considers three main approaches: crowd video analysis, crowd spatio-temporal analysis, and crowd social media analysis. The key research contributions in each approach are presented, and the most significant techniques and algorithms used to improve the precision of results that could be integrated into solutions to enhance the quality of services in a smart city are analyzed.

[1]  Bingbing Ni,et al.  Recurrent Modeling of Interaction Context for Collective Activity Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Duan-Yu Chen,et al.  Dynamic human crowd modeling and its application to anomalous events detcetion , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[3]  Jon Atli Benediktsson,et al.  Spatial technology and social media in remote sensing: challenges and opportunities [point of view] , 2017, Proc. IEEE.

[4]  Krzysztof Janowicz,et al.  Can Twitter data be used to validate travel demand models , 2015 .

[5]  Silvio Savarese,et al.  Social Scene Understanding: End-to-End Multi-person Action Localization and Collective Activity Recognition , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Ciro Cattuto,et al.  Predicting human mobility through the assimilation of social media traces into mobility models , 2016, EPJ Data Science.

[7]  Limin Wang,et al.  Latent Hierarchical Model of Temporal Structure for Complex Activity Classification , 2014, IEEE Transactions on Image Processing.

[8]  Bin Wang,et al.  Spatio-Temporal Variable Selection Based Support Vector Regression for Urban Traffic Flow Prediction , 2014 .

[9]  Mu-Chen Chen,et al.  Forecasting the short-term metro passenger flow with empirical mode decomposition and neural networks , 2012 .

[10]  Song-Chun Zhu,et al.  CERN: Confidence-Energy Recurrent Network for Group Activity Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Wanggen Wan,et al.  Using Location-Based Social Media Data to Observe Check-In Behavior and Gender Difference: Bringing Weibo Data into Play , 2018, ISPRS Int. J. Geo Inf..

[12]  Ye Tao,et al.  Abnormal Prediction of Dense Crowd Videos by a Purpose–Driven Lattice Boltzmann Model , 2017, Int. J. Appl. Math. Comput. Sci..

[13]  Ricardo Silva,et al.  Predicting traffic volumes and estimating the effects of shocks in massive transportation systems , 2015, Proceedings of the National Academy of Sciences.

[14]  Licia Capra,et al.  Urban Computing: Concepts, Methodologies, and Applications , 2014, TIST.

[15]  François Brémond,et al.  Human violence recognition and detection in surveillance videos , 2016, 2016 13th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[16]  SogaKenichi,et al.  Pedestrian monitoring techniques for crowd-flow prediction , 2017 .

[17]  Jure Leskovec,et al.  Friendship and mobility: user movement in location-based social networks , 2011, KDD.

[18]  Wei Liu,et al.  Network-wide Crowd Flow Prediction of Sydney Trains via Customized Online Non-negative Matrix Factorization , 2018, CIKM.

[19]  Fan Zhang,et al.  Spatio-Temporal Analysis of Passenger Travel Patterns in Massive Smart Card Data , 2017, IEEE Transactions on Intelligent Transportation Systems.

[20]  Xuan Song,et al.  CityMomentum: an online approach for crowd behavior prediction at a citywide level , 2015, UbiComp.

[21]  Juan Carlos García-Palomares,et al.  Social media and urban mobility: Using twitter to calculate home-work travel matrices , 2019, Cities.

[22]  Jon Atli Benediktsson,et al.  Big Data for Remote Sensing: Challenges and Opportunities , 2016, Proceedings of the IEEE.

[23]  Muhammad Zia,et al.  Single- and two-person action recognition based on silhouette shape and optical point descriptors , 2018, Signal Image Video Process..

[24]  Imran N. Junejo,et al.  Social network model for crowd anomaly detection and localization , 2017, Pattern Recognit..

[25]  Biao Leng,et al.  A novel wavelet-SVM short-time passenger flow prediction in Beijing subway system , 2015, Neurocomputing.

[26]  Florian Baumann.  Action Recognition with HOG-OF Features , 2013, GCPR.

[27]  Xianfeng Tang,et al.  Modeling Spatial-Temporal Dynamics for Traffic Prediction , 2018, ArXiv.

[28]  José J. Ramasco,et al.  Exploring the potential of phone call data to characterize the relationship between social network and travel behavior , 2015 .

[29]  Petros A. Ioannou,et al.  Traffic Flow Prediction for Road Transportation Networks With Limited Traffic Data , 2015, IEEE Transactions on Intelligent Transportation Systems.

[30]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[31]  Stefania Bandini,et al.  Modeling a Crowd of Groups: Multidisciplinary and Methodological Challenges , 2013, Modeling, Simulation and Visual Analysis of Crowds.

[32]  Takeo Kanade,et al.  Tracking in unstructured crowded scenes , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[33]  Ren Gang,et al.  SAE for the prediction of road traffic status from taxicab operating data and bus smart card data , 2017 .

[34]  Vassilis Kostakos,et al.  Towards Real-time Emergency Response using Crowd Supported Analysis of Social Media , 2011 .

[35]  Claudio Bettini,et al.  Incremental release of differentially-private check-in data , 2015, PERCOM 2015.

[36]  Fei-Fei Li,et al.  Socially-Aware Large-Scale Crowd Forecasting , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Feng Liu,et al.  Crowd Flow Prediction by Deep Spatio-Temporal Transfer Learning , 2018, ArXiv.

[38]  Wang-Chien Lee,et al.  Semantic trajectory mining for location prediction , 2011, GIS.

[39]  Tieniu Tan,et al.  A survey on visual surveillance of object motion and behaviors , 2004, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[40]  Zhang Xiong,et al.  Probability tree based passenger flow prediction and its application to the Beijing subway system , 2013, Frontiers of Computer Science.

[41]  Ambuj K. Singh,et al.  FCCF: forecasting citywide crowd flows based on big data , 2016, SIGSPATIAL/GIS.

[42]  Tianrui Li,et al.  ST-MVL: Filling Missing Values in Geo-Sensory Time Series Data , 2016, IJCAI.

[43]  Stefania Bandini,et al.  Studying Pedestrian and Crowd Dynamics through Integrated Analysis and Synthesis , 2013, IEEE Intelligent Systems.

[44]  Anthony K. H. Tung,et al.  SMiLer: A Semi-Lazy Time Series Prediction System for Sensors , 2015, SIGMOD Conference.

[45]  Fan Zhang,et al.  PULSE: A Real Time System for Crowd Flow Prediction at Metropolitan Subway Stations , 2016, ECML/PKDD.

[46]  Stavros J. Perantonis,et al.  Detecting abnormal human behaviour using multiple cameras , 2009, Signal Process..

[47]  Haihong Zhu,et al.  Explorative Analysis of Wuhan Intra-Urban Human Mobility Using Social Media Check-In Data , 2015, PloS one.

[48]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Billy M. Williams,et al.  Modeling and Forecasting Vehicular Traffic Flow as a Seasonal ARIMA Process: Theoretical Basis and Empirical Results , 2003, Journal of Transportation Engineering.

[50]  Shaohua Wang,et al.  Extraction and monitoring approach of dynamic urban commercial area using check-in data from Weibo , 2019, Sustainable Cities and Society.

[51]  Hansi Senaratne,et al.  Moving on Twitter: using episodic hotspot and drift analysis to detect and characterise spatial trajectories , 2014, LBSN '14.

[52]  Qing He,et al.  Forecasting the Subway Passenger Flow Under Event Occurrences With Social Media , 2017, IEEE Transactions on Intelligent Transportation Systems.

[53]  T. Rashidi,et al.  Exploring the capacity of social media data for modelling travel behaviour: Opportunities and challenges , 2017 .

[54]  Bart de Boer,et al.  Simulating and Controlling Civilian Crowds in Robocup Rescue , 2005 .

[55]  Ying Jin,et al.  Pedestrian monitoring techniques for crowd-flow prediction , 2017 .

[56]  Stefan Stieglitz,et al.  Social media analytics - Challenges in topic discovery, data collection, and data preparation , 2018, Int. J. Inf. Manag..

[57]  Lun Wu,et al.  Intra-Urban Human Mobility and Activity Transition: Evidence from Social Media Check-In Data , 2014, PloS one.

[58]  Xiuwen Yi,et al.  DNN-based prediction model for spatio-temporal data , 2016, SIGSPATIAL/GIS.

[59]  Yu Zheng,et al.  Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction , 2016, AAAI.

[60]  Greg Mori,et al.  A Hierarchical Deep Temporal Model for Group Activity Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Soraia Raupp Musse,et al.  Crowd Analysis Using Computer Vision Techniques , 2010, IEEE Signal Processing Magazine.

[62]  Yu Zheng,et al.  Traffic prediction in a bike-sharing system , 2015, SIGSPATIAL/GIS.

[63]  Ahmad Nickabadi,et al.  A Multi-Stream Convolutional Neural Network Framework for Group Activity Recognition , 2018, ArXiv.

[64]  Katia P. Sycara,et al.  Nonnegative Matrix Tri-Factorization with Graph Regularization for Community Detection in Social Networks , 2015, IJCAI.

[65]  Yong Gao,et al.  Uncovering Patterns of Inter-Urban Trip and Spatial Interaction from Social Media Check-In Data , 2013, PloS one.