Crowd emotion evaluation based on fuzzy inference of arousal and valence

Abstract Crowd behavior analysis is an important research topic in the field of video surveillance and public safety management. Crowd emotion has a strong relation with the crowd behavior. However, it is very challenging to predict crowd emotion using conventional emotion clues of humans from the video surveillance data, e.g. facial expression or body gesture. To tackle this challenge, this paper presents a crowd emotion evaluation method using fuzzy inference according to the arousal-valence model of the crowd movement. Specifically, the enthalpy, magnitude variance, confusion index and crowd density are extracted to describe crowd emotion. The enthalpy value and magnitude variance are taken as the input of the fuzzy inference system of arousal. And the confusion index and crowd density are used as the input of the fuzzy system of valence. The arousal value and valence value are the output respectively. Through establishing the relationship between arousal, valence and crowd features, the fuzzy rules are constructed to infer the emotion in the crowd scene. Experimental results show that the proposed method can effectively evaluate the arousal and valence in crowd emotion.

[1]  Hong Zhang,et al.  Simulation of queuing time in crowd evacuation by discrete time loss queuing method , 2019, International Journal of Modern Physics C.

[2]  Worapan Kusakunniran,et al.  Localization and Classification of Rice-grain Images Using Region Proposals-based Convolutional Neural Network , 2020, Int. J. Autom. Comput..

[3]  Ofir Turel,et al.  Social networking sites use and the morphology of a social-semantic brain network , 2018, Social neuroscience.

[4]  Wentong Cai,et al.  A data-driven path planning model for crowd capacity analysis , 2019, J. Comput. Sci..

[5]  Jeffrey V. Nickerson,et al.  Arousal, valence, and volume: how the influence of online review characteristics differs with respect to utilitarian and hedonic products , 2018, Eur. J. Inf. Syst..

[6]  Guijuan Zhang,et al.  Learning crowd behavior from real data: A residual network method for crowd simulation , 2020, Neurocomputing.

[7]  Huimin Ma,et al.  Semantic Head Enhanced Pedestrian Detection in a Crowd , 2019, Neurocomputing.

[8]  Stuart I. Brown.  A personal perspective on the work of Jaak Panksepp , 2020 .

[9]  A. Khosla,et al.  Visual Object Tracking by Fusion of Audio Imaging in Template Matching Framework , 2019, International Journal of Image, Graphics and Signal Processing.

[10]  Marc Van Droogenbroeck,et al.  ViBe: A Universal Background Subtraction Algorithm for Video Sequences , 2011, IEEE Transactions on Image Processing.

[11]  Zidong Wang,et al.  A Dynamic Neighborhood-Based Switching Particle Swarm Optimization Algorithm , 2020, IEEE Transactions on Cybernetics.

[12]  R. Thayer.  The biopsychology of mood and arousal , 1989 .

[13]  Rongrong Ji,et al.  Exploring Coherent Motion Patterns via Structured Trajectory Learning for Crowd Mood Modeling , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Peng Wang,et al.  MobileCount: An efficient encoder-decoder framework for real-time crowd counting , 2020, Neurocomputing.

[15]  Yinfeng Fang,et al.  Detection of Salient Crowd Motion Based on Repulsive Force Network and Direction Entropy , 2019, Entropy.

[16]  Chabane Djeraba,et al.  Real-time crowd motion analysis , 2008, 2008 19th International Conference on Pattern Recognition.

[17]  Sriparna Saha,et al.  Towards Sentiment-Aware Multi-Modal Dialogue Policy Learning , 2020, Cognitive Computation.

[18]  Amol Patwardhan,et al.  Edge Based Grid Super-Imposition for Crowd Emotion Recognition , 2016, ArXiv.

[19]  Qi Li,et al.  Robust Object Tracking via Information Theoretic Measures , 2020, International Journal of Automation and Computing.

[20]  M. Nazari,et al.  Droplet size prediction in a microfluidic flow focusing device using an adaptive network based fuzzy inference system , 2020, Biomedical Microdevices.

[21]  Huijun Di,et al.  SCLNet: Spatial context learning network for congested crowd counting , 2020, Neurocomputing.

[22]  Francisco Herrera,et al.  Revisiting crowd behaviour analysis through deep learning: Taxonomy, anomaly detection, crowd emotions, datasets, opportunities and prospects , 2020, Information Fusion.

[23]  R. Venkatesh Babu,et al.  Anomaly detection via short local trajectories , 2017, Neurocomputing.

[24]  Hui Yu,et al.  Real-Time Facial Affective Computing on Mobile Devices , 2020, Sensors.

[25]  Fuad E. Alsaadi,et al.  A novel randomised particle swarm optimizer , 2020, Int. J. Mach. Learn. Cybern..

[26]  Anthony G. Hudetz,et al.  Functional and Topological Conditions for Explosive Synchronization Develop in Human Brain Networks with the Onset of Anesthetic-Induced Unconsciousness , 2016, Front. Comput. Neurosci..

[27]  Differences in adult and adolescent listeners’ ratings of valence and arousal in emotional prosody , 2018, Cognition & emotion.

[28]  Ayaz Ahmad,et al.  Density independent hydrodynamics model for crowd coherency detection , 2017, Neurocomputing.

[29]  Luciano Telesca,et al.  Transportation hazard spatial analysis using crowd-sourced social network data , 2019, Physica A: Statistical Mechanics and its Applications.

[30]  Yuan Yuan,et al.  A Novel Sigmoid-Function-Based Adaptive Weighted Particle Swarm Optimizer , 2019, IEEE Transactions on Cybernetics.

[31]  Arindam Sikdar,et al.  An adaptive training-less framework for anomaly detection in crowd scenes , 2020, Neurocomputing.

[32]  Bin Liang,et al.  Motion recognition based on Kinect for human-computer intelligent interaction , 2019, Journal of Physics: Conference Series.

[33]  Nicu Sebe,et al.  Emotion-Based Crowd Representation for Abnormality Detection , 2016, ArXiv.

[34]  Li Wang,et al.  Crowd Counting Network with Self-attention Distillation , 2020 .

[35]  Zhipeng Wang,et al.  Anomaly detection in crowded scenes using motion energy model , 2017, Multimedia Tools and Applications.

[36]  Wei Zhang,et al.  Two-branch fusion network with attention map for crowd counting , 2020, Neurocomputing.

[37]  I. Ketut Eddy Purnama,et al.  Multi agent with multi behavior based on particle swarm optimization (PSO) for crowd movement in fire evacuation , 2013, 2013 Fourth International Conference on Intelligent Control and Information Processing (ICICIP).

[38]  Fei-Yue Wang,et al.  Accurate and robust eye center localization via fully convolutional networks , 2019, IEEE/CAA Journal of Automatica Sinica.

[39]  Qi Wang,et al.  Online Anomaly Detection in Crowd Scenes via Structure Analysis , 2015, IEEE Transactions on Cybernetics.

[40]  Jane Anderson.  Damasio’s body-map-based view, Panksepp’s affect-centric view, and the evolutionary advantages of consciousness , 2019, South African Journal of Philosophy.

[41]  Ming Zhu,et al.  Attentive multi-stage convolutional neural network for crowd counting , 2020, Pattern Recognit. Lett..

[42]  Jian Ma,et al.  Dynamics of emotional contagion in dense pedestrian crowds , 2020 .

[43]  Matthias Rauterberg,et al.  A Hierarchical Bayesian Model for Crowd Emotions , 2016, Front. Comput. Neurosci..

[44]  Brett Stevens,et al.  Scene perception guided crowd anomaly detection , 2020, Neurocomputing.

[45]  Yue Ming,et al.  Deep learning for monocular depth estimation: A review , 2021, Neurocomputing.

[46]  J. Kanter,et al.  Using the primary process emotional-behavioural system to better meet patient needs in psychotherapy. , 2019, Clinical psychology & psychotherapy.

[47]  The role and the meaning of work in Freud's cultural writings , 2012 .

[48]  Zhen He,et al.  Crowd panic state detection using entropy of the distribution of enthalpy , 2019, Physica A: Statistical Mechanics and its Applications.

[49]  Tehreem Qasim,et al.  A low dimensional descriptor for detection of anomalies in crowd videos , 2019, Math. Comput. Simul..

[50]  Hui Yu,et al.  Physics Inspired Methods for Crowd Video Surveillance and Analysis: A Survey , 2018, IEEE Access.

[51]  Jiliu Zhou,et al.  Dual-channel CNN for efficient abnormal behavior identification through crowd feature engineering , 2018, Machine Vision and Applications.

[52]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[53]  A. Khosla,et al.  Crowd Escape Event Detection via Pooling Features of Optical Flow for Intelligent Video Surveillance Systems , 2019, International Journal of Image, Graphics and Signal Processing.

[54]  Yuke Li,et al.  A Deep Spatiotemporal Perspective for Understanding Crowd Behavior , 2018, IEEE Transactions on Multimedia.

[55]  Dong-Ming Yan,et al.  Parallel Computation of 3D Clipped Voronoi Diagrams. , 2020, IEEE transactions on visualization and computer graphics.

[56]  Min Zhou,et al.  Guided crowd evacuation: approaches and challenges , 2019, IEEE/CAA Journal of Automatica Sinica.

[57]  Xiaohui Liu,et al.  An N-State Markovian Jumping Particle Swarm Optimization Algorithm , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[58]  Man Lv,et al.  Facial Expression Recognition Based on a Hybrid Model Combining Deep and Shallow Features , 2019, Cognitive Computation.

[59]  Qian Zhang,et al.  Energy Level-Based Abnormal Crowd Behavior Detection , 2018, Sensors.

[60]  Yuan Zhou,et al.  Adversarial Learning for Multiscale Crowd Counting Under Complex Scenes , 2020, IEEE Transactions on Cybernetics.

[61]  Xuguang Zhang,et al.  Crowd Abnormal Event Detection Based on Sparse Coding , 2019, Int. J. Humanoid Robotics.

[62]  M. Solms.  Depression: A neuropsychoanalytic perspective , 2012 .

[63]  Zidong Wang,et al.  A Novel Particle Swarm Optimization Approach for Patient Clustering From Emergency Departments , 2019, IEEE Transactions on Evolutionary Computation.

[64]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[65]  Wu He,et al.  An emotion based simulation framework for complex evacuation scenarios , 2019, Graph. Model..