On Real-time Detecting Passenger Flow Anomalies

In large and medium-sized cities, detecting unusual changes of crowds of people on the streets is needed for public security, transportation management, emergency control, and terrorism prevention. As public transportation has the capability to bring a large number of people to an area in a short amount of time, real-time discovery of anomalies in passenger numbers is an effective way to detect crowd anomalies. In this paper, we devise an approach called Kochab. Kochab adopts a generative model and combines the prior knowledge about passenger flows. Hence, it can detect anomalies in the numbers of incoming and outgoing passengers within a certain time and spatial area, including anomalous events along with their durations and severities. Through well-designed inference algorithms, Kochab requires only a moderate amount of historical data to be sample data. As such, Kochab shows good performance in real time and makes prompt responses to user' s interactive analysis requests. In particular, based on the recognized anomalous events, we capture event patterns which give us hints to link to activities or status in cities. In addition, for the convenience of method evaluation and comparison, we create an open Stream Anomaly Benchmark on the basis of large-scale real-world data. This benchmark will prove useful for other researchers too. Using this benchmark, we compare Kochab with four other methods. The experimental results show that Kochab is sensitive to population flow anomalies and has superior accuracy in detecting anomalies in terms of precision, recall and the F1 score.

[1]  Le Gruenwald,et al.  Research issues in outlier detection for data streams , 2014, SKDD.

[2]  Eamonn J. Keogh,et al.  Finding the most unusual time series subsequence: algorithms and applications , 2006, Knowledge and Information Systems.

[3]  Subutai Ahmad,et al.  Unsupervised real-time anomaly detection for streaming data , 2017, Neurocomputing.

[4]  Saeed Amizadeh,et al.  Generic and Scalable Framework for Automated Time-series Anomaly Detection , 2015, KDD.

[5]  Lin Sun,et al.  iBOAT: Isolation-Based Online Anomalous Trajectory Detection , 2013, IEEE Transactions on Intelligent Transportation Systems.

[6]  Lovekesh Vig,et al.  Long Short Term Memory Networks for Anomaly Detection in Time Series , 2015, ESANN.

[7]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[8]  Sanjay Chawla,et al.  Inferring the Root Cause in Road Traffic Anomalies , 2012, 2012 IEEE 12th International Conference on Data Mining.

[9]  Junshui Ma,et al.  Online novelty detection on temporal sequences , 2003, KDD '03.

[10]  Padhraic Smyth,et al.  Adaptive event detection with time-varying poisson processes , 2006, KDD '06.

[11]  Brian Litt,et al.  One-Class Novelty Detection for Seizure Analysis from Intracranial EEG , 2006, J. Mach. Learn. Res..

[12]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[13]  Christos Faloutsos,et al.  Modeling Temporal Activity to Detect Anomalous Behavior in Social Media , 2017, ACM Trans. Knowl. Discov. Data.

[14]  Charu C. Aggarwal,et al.  Outlier Detection for Temporal Data: A Survey , 2014, IEEE Transactions on Knowledge and Data Engineering.

[15]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[16]  S. L. Scott Bayesian Methods for Hidden Markov Models , 2002 .

[17]  Stephen L. Scott,et al.  Bayesian Methods and Extensions for the Two State Markov Modulated Poisson Process , 1998 .

[18]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[19]  Qi Liu,et al.  Unsupervised detection of contextual anomaly in remotely sensed data , 2017 .

[20]  Nuno Constantino Castro,et al.  Time Series Data Mining , 2009, Encyclopedia of Database Systems.

[21]  Hui Xiong,et al.  Catch Me If You Can: Detecting Pickpocket Suspects from Large-Scale Transit Records , 2016, KDD.