A Diffusion and Clustering-Based Approach for Finding Coherent Motions and Understanding Crowd Scenes

This paper addresses the problem of detecting coherent motions in crowd scenes and presents its two applications in crowd scene understanding: semantic region detection and recurrent activity mining. It processes input motion fields (e.g., optical flow fields) and produces a coherent motion field named thermal energy field. The thermal energy field is able to capture both motion correlation among particles and the motion trends of individual particles, which are helpful to discover coherency among them. We further introduce a two-step clustering process to construct stable semantic regions from the extracted time-varying coherent motions. These semantic regions can be used to recognize pre-defined activities in crowd scenes. Finally, we introduce a cluster-and-merge process, which automatically discovers recurrent activities in crowd scenes by clustering and merging the extracted coherent motions. Experiments on various videos demonstrate the effectiveness of our approach.

[1]  Hongyuan Zha,et al.  Inferring User Image-Search Goals Under the Implicit Guidance of Users , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[2]  Qingshan Liu,et al.  Abnormal detection using interaction energy potentials , 2011, CVPR 2011.

[3]  Mubarak Shah,et al.  A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Xiaogang Wang,et al.  Random field topic model for semantic region analysis in crowded scenes from tracklets , 2011, CVPR 2011.

[5]  Jitendra Malik,et al.  Object Segmentation by Long Term Analysis of Point Trajectories , 2010, ECCV.

[6]  Mário A. T. Figueiredo,et al.  Trajectory Classification Using Switched Dynamical Hidden Markov Models , 2010, IEEE Transactions on Image Processing.

[7]  Wilhelm Burger,et al.  Digital Image Processing - An Algorithmic Introduction using Java , 2008, Texts in Computer Science.

[8]  Tao Mei,et al.  Finding perfect rendezvous on the go: accurate mobile visual localization and its applications to routing , 2012, ACM Multimedia.

[9]  Mubarak Shah,et al.  Abnormal crowd behavior detection using social force model , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[10]  Guillermo Sapiro,et al.  Robust anisotropic diffusion , 1998, IEEE Trans. Image Process..

[11]  Yongyi Yang,et al.  Optical Flow Estimation for a Periodic Image Sequence , 2010, IEEE Transactions on Image Processing.

[12]  Wei Zeng,et al.  Single and Multiple View Detection, Tracking and Video Analysis in Crowded Environments , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[13]  Sergio A. Velastin,et al.  Crowd analysis: a survey , 2008, Machine Vision and Applications.

[14]  Qi Tian,et al.  Multimedia search reranking: A literature survey , 2014, CSUR.

[15]  Mohan M. Trivedi,et al.  Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Joachim Weickert,et al.  Anisotropic diffusion in image processing , 1996 .

[17]  Rachid Deriche,et al.  Universität Des Saarlandes Fachrichtung 6.1 – Mathematik Colour, Texture, and Motion in Level Set Based Segmentation and Tracking Colour, Texture, and Motion in Level Set Based Segmentation and Tracking , 2022 .

[18]  Shaogang Gong,et al.  Multi-camera activity correlation analysis , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[19]  Shaogang Gong,et al.  Scene Segmentation for Behaviour Correlation , 2008, ECCV.

[20]  Daniel Cremers,et al.  Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation , 2005, International Journal of Computer Vision.

[21]  Yunde Jia,et al.  Adaptive diffusion flow active contours for image segmentation , 2013, Comput. Vis. Image Underst..

[22]  Herbert Edelsbrunner,et al.  Incremental topological flipping works for regular triangulations , 1992, SCG '92.

[23]  L. Rosenhead.  Conduction of Heat in Solids , 1947, Nature.

[24]  Jianxin Wu,et al.  Towards Good Practices for Action Video Encoding , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Xiaogang Wang,et al.  Measuring Crowd Collectiveness , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Mubarak Shah,et al.  Learning motion patterns in crowded scenes using motion flow field , 2008, 2008 19th International Conference on Pattern Recognition.

[27]  Xiaogang Wang,et al.  Coherent Filtering: Detecting Coherent Motions from Crowd Clutters , 2012, ECCV.

[28]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[29]  Hau-San Wong,et al.  Crowd Motion Partitioning in a Scattered Motion Field , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[30]  Yasuyuki Matsushita,et al.  Motion detail preserving optical flow estimation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[31]  Dahua Lin,et al.  Learning visual flows: A Lie algebraic approach , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  Jianxin Wu,et al.  Finding Coherent Motions and Semantic Regions in Crowd Scenes: A Diffusion and Clustering Approach , 2014, ECCV.

[33]  W. Eric L. Grimson,et al.  Trajectory analysis and semantic region modeling using a nonparametric Bayesian model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[34]  Jean-Marc Odobez,et al.  Temporal Analysis of Motif Mixtures Using Dirichlet Processes , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Jean-Marc Odobez,et al.  A Sequential Topic Model for Mining Recurrent Activities from Long Term Video Logs , 2013, International Journal of Computer Vision.

[36]  W. Eric L. Grimson,et al.  Trajectory Analysis and Semantic Region Modeling Using Nonparametric Hierarchical Bayesian Models , 2011, International Journal of Computer Vision.

[37]  Jorge L. C. Sanz,et al.  Optical flow computation using extended constraints , 1996, IEEE Trans. Image Process..

[38]  Joachim Weickert,et al.  Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods , 2005, International Journal of Computer Vision.

[39]  Zhongfei Zhang,et al.  An Incremental DPMM-Based Method for Trajectory Clustering, Modeling, and Retrieval , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.