Understanding vehicular traffic behavior from video: a survey of unsupervised approaches

Abstract. Recent emerging trends for automatic behavior analysis and understanding from infrastructure video are reviewed. Research has shifted from high-resolution estimation of vehicle state and instead, pushed machine learning approaches to extract meaningful patterns in aggregates in an unsupervised fashion. These patterns represent priors on observable motion, which can be utilized to describe a scene, answer behavior questions such as where is a vehicle going, how many vehicles are performing the same action, and to detect an abnormal event. The review focuses on two main methods for scene description, trajectory clustering and topic modeling. Example applications that utilize the behavioral modeling techniques are also presented. In addition, the most popular public datasets for behavioral analysis are presented. Discussion and comment on future directions in the field are also provided.

[1]  Zhongfei Zhang,et al.  An Incremental DPMM-Based Method for Trajectory Clustering, Modeling, and Retrieval , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Mohan M. Trivedi,et al.  A Survey of Vision-Based Trajectory Learning and Analysis for Surveillance , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[3]  Jiang Wu,et al.  E3TP: A Novel Trajectory Prediction Algorithm in Moving Objects Databases , 2009, PAISI.

[4]  Tarek Sayed,et al.  Large-Scale Automated Analysis of Vehicle Interactions and Collisions , 2010 .

[5]  Thomas Hofmann,et al.  Probabilistic Latent Semantic Analysis , 1999, UAI.

[6]  Jean-Marc Odobez,et al.  Bridging the past, present and future: Modeling scene activities from event relationships and global rules , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Hanqing Lu,et al.  Learning Semantic Motion Patterns for Dynamic Scenes by Improved Sparse Topical Coding , 2012, 2012 IEEE International Conference on Multimedia and Expo.

[8]  Aggelos K. Katsaggelos,et al.  Anomalous video event detection using spatiotemporal context , 2011 .

[9]  Shaogang Gong,et al.  Video Behaviour Mining Using a Dynamic Topic Model , 2011, International Journal of Computer Vision.

[10]  Elisa Ricci,et al.  Earth mover's prototypes: A convex learning approach for discovering activity patterns in dynamic scenes , 2011, CVPR 2011.

[11]  Shaogang Gong,et al.  Stream-based joint exploration-exploitation active learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Mohan M. Trivedi,et al.  Learning, Modeling, and Classification of Vehicle Track Patterns from Live Video , 2008, IEEE Transactions on Intelligent Transportation Systems.

[13]  Sergio A. Velastin,et al.  A Review of Computer Vision Techniques for the Analysis of Urban Traffic , 2011, IEEE Transactions on Intelligent Transportation Systems.

[14]  Xiaogang Wang,et al.  Action Recognition Using Topic Models , 2011, Visual Analysis of Humans.

[15]  W. Eric L. Grimson,et al.  Trajectory Analysis and Semantic Region Modeling Using Nonparametric Hierarchical Bayesian Models , 2011, International Journal of Computer Vision.

[16]  Mubarak Shah,et al.  Scene understanding by statistical modeling of motion patterns , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Jitendra Malik,et al.  Spectral grouping using the Nystrom method , 2004, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Tieniu Tan,et al.  A system for learning statistical motion patterns , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Shaogang Gong,et al.  Stream-Based Active Unusual Event Detection , 2010, ACCV.

[20]  Kunfeng Wang,et al.  Video processing techniques for traffic flow monitoring: A survey , 2011, 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC).

[21]  Mohan M. Trivedi,et al.  Trajectory Learning for Activity Understanding: Unsupervised, Multilevel, and Long-Term Adaptive Approach , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Zhouyu Fu,et al.  Semantic-Based Surveillance Video Retrieval , 2007, IEEE Transactions on Image Processing.

[23]  Gwenn Englebienne,et al.  A Non-parametric Hierarchical Model to Discover Behavior Dynamics from Tracks , 2012, ECCV.

[24]  W. Eric L. Grimson,et al.  Trajectory analysis and semantic region modeling using a nonparametric Bayesian model , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  William W. Hunter,et al.  Pedestrian and Bicyclist Intersection Safety Indices: Final Report , 2006 .

[26]  Tim J. Ellis,et al.  Learning semantic scene models from observing activity in visual surveillance , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[27]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[28]  Christian Laugier,et al.  Incremental Learning of Statistical Motion Patterns With Growing Hidden Markov Models , 2007, IEEE Transactions on Intelligent Transportation Systems.

[29]  Shaogang Gong,et al.  Detecting and discriminating behavioural anomalies , 2011, Pattern Recognit..

[30]  Anna Vilanova,et al.  Evaluation of fiber clustering methods for diffusion tensor imaging , 2005, VIS 05. IEEE Visualization, 2005..

[31]  Meng Wang,et al.  Automatic adaptation of a generic pedestrian detector to a specific traffic scene , 2011, CVPR 2011.

[32]  Osama Masoud,et al.  Learning Traffic Patterns at Intersections by Spectral Clustering of Motion Trajectories , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Nicu Sebe,et al.  Exploiting Sparse Representations for Robust Analysis of Noisy Complex Video Scenes , 2012, ECCV.

[34]  Luc Van Gool,et al.  What's going on? Discovering spatio-temporal dependencies in dynamic scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Mubarak Shah,et al.  Multi feature path modeling for video surveillance , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[36]  Tao Xiang,et al.  Delta-Dual Hierarchical Dirichlet Processes: A pragmatic abnormal behaviour detector , 2011, 2011 International Conference on Computer Vision.

[37]  Hyeran Byun,et al.  Detection of dominant flow and abnormal events in surveillance video , 2011 .

[38]  Svetha Venkatesh,et al.  Large-scale statistical modeling of motion patterns: a Bayesian nonparametric approach , 2012, ICVGIP '12.

[39]  Jean-Marc Odobez,et al.  A Sequential Topic Model for Mining Recurrent Activities from Long Term Video Logs , 2013, International Journal of Computer Vision.

[40]  Christian Laugier,et al.  Modelling Dynamic Scenes at Unsignalised Road Intersections , 2011 .

[41]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[42]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[43]  Jean-Marc Odobez,et al.  Topic models for scene analysis and abnormality detection , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[44]  W. Eric L. Grimson,et al.  Unsupervised Activity Perception by Hierarchical Bayesian Models , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  W. Eric L. Grimson,et al.  Unsupervised Activity Perception in Crowded and Complicated Scenes Using Hierarchical Bayesian Models , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Mohan M. Trivedi,et al.  Learning trajectory patterns by clustering: Experimental studies and comparative evaluation , 2009, CVPR.

[47]  Zhongke Shi,et al.  Understanding dynamic scenes by hierarchical motion pattern mining , 2011, 2011 IEEE International Conference on Multimedia and Expo.

[48]  Henry A. Kautz,et al.  Learning and inferring transportation routines , 2004, Artif. Intell..

[49]  Mohan M. Trivedi,et al.  Real-Time Video-Based Traffic Measurement and Visualization System for Energy/Emissions , 2012, IEEE Transactions on Intelligent Transportation Systems.

[50]  Tieniu Tan,et al.  Comparison of Similarity Measures for Trajectory Clustering in Outdoor Surveillance Scenes , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[51]  Gian Luca Foresti,et al.  On-line trajectory clustering for anomalous events detection , 2006, Pattern Recognit. Lett..

[52]  Nicolas Saunier,et al.  Motion Prediction Methods for Surrogate Safety Analysis , 2013 .

[53]  Shaogang Gong,et al.  Learning Behavioural Context , 2012, International Journal of Computer Vision.

[54]  Nicoletta Noceti,et al.  Learning common behaviors from large sets of unlabeled temporal series , 2012, Image Vis. Comput..

[55]  H. Kuhn The Hungarian method for the assignment problem , 1955 .