Anomaly detection in spatiotemporal data via regularized non-negative tensor analysis

Anomaly detection in multidimensional data is a challenging task. Detecting anomalous mobility patterns in a city needs to take spatial, temporal, and traffic information into consideration. Although existing techniques are able to extract spatiotemporal features for anomaly analysis, few systematic analysis about how different factors contribute to or affect the anomalous patterns has been proposed. In this paper, we propose a novel technique to localize spatiotemporal anomalous events based on tensor decomposition. The proposed method employs a spatial-feature-temporal tensor model and analyzes latent mobility patterns through unsupervised learning. We first train the model based on historical data and then use the model to capture the anomalies, i.e., the mobility patterns that are significantly different from the normal patterns. The proposed technique is evaluated based on the yellow-cab dataset collected from New York City. The results show several interesting latent mobility patterns and traffic anomalies that can be deemed as anomalous events in the city, suggesting the effectiveness of the proposed anomaly detection method.

[1]  Christos Faloutsos,et al.  Spotting misbehaviors in location-based social networks using tensors , 2014, WWW.

[2]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD 2000.

[3]  Thomas S. Huang,et al.  One-class SVM for learning in image retrieval , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[4]  Chao Li,et al.  Discovering Urban Spatio-temporal Structure from Time-Evolving Traffic Networks , 2014, APWeb.

[5]  Christos Faloutsos,et al.  Suspicious Behavior Detection: Current Trends and Future Directions , 2016, IEEE Intelligent Systems.

[6]  Weiwei Guo,et al.  Higher rank Support Tensor Machines for visual recognition , 2012, Pattern Recognit..

[7]  Dongyu Liu,et al.  SmartAdP: Visual Analytics of Large-scale Taxi Trajectories for Selecting Billboard Locations , 2017, IEEE Transactions on Visualization and Computer Graphics.

[8]  Yingcai Wu,et al.  A Survey of Visual Analytics Techniques and Applications: State-of-the-Art Research and Future Challenges , 2013, Journal of Computer Science and Technology.

[9]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[10]  Haesun Park,et al.  Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework , 2014, J. Glob. Optim..

[11]  Hadi Fanaee-T,et al.  Event and Anomaly Detection Using Tucker3 Decomposition , 2014, ArXiv.

[12]  Amos Storkey,et al.  Advances in Neural Information Processing Systems 20 , 2007 .

[13]  Hadi Fanaee-T,et al.  An eigenvector-based hotspot detection , 2014, ArXiv.

[14]  Hui Zhang,et al.  iTTVis: Interactive Visualization of Table Tennis Data , 2018, IEEE Transactions on Visualization and Computer Graphics.

[15]  Lei Shi,et al.  STenSr: Spatio-temporal tensor streams for anomaly detection and pattern discovery , 2015, Knowledge and Information Systems.

[16]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[17]  Nikos D. Sidiropoulos,et al.  ParCube: Sparse Parallelizable Tensor Decompositions , 2012, ECML/PKDD.

[18]  Steffen Rendle,et al.  Factorization Machines with libFM , 2012, TIST.

[19]  Yingcai Wu,et al.  HomeFinder Revisited: Finding Ideal Homes with Reachability-Centric Multi-Criteria Decision Making , 2018, CHI.

[20]  Zhe Zhao,et al.  Improving User Topic Interest Profiles by Behavior Factorization , 2015, WWW.

[21]  Sergio García,et al.  DIMENSIONALITY REDUCTION FOR DAMAGE DETECTION IN ENGINEERING STRUCTURES , 2012 .

[22]  Jimeng Sun,et al.  Beyond streams and graphs: dynamic tensor analysis , 2006, KDD '06.

[23]  J. Macgregor,et al.  Monitoring batch processes using multiway principal component analysis , 1994 .

[24]  Lars Schmidt-Thieme,et al.  Factorization Models for Forecasting Student Performance , 2011, EDM.

[25]  David S. Ebert,et al.  DimScanner: A relation-based visual exploration approach towards data dimension inspection , 2016, 2016 IEEE Conference on Visual Analytics Science and Technology (VAST).

[26]  Hadi Fanaee-T,et al.  EigenEvent: An algorithm for event detection from complex data streams in syndromic surveillance , 2014, Intell. Data Anal..

[27]  Jing Yang,et al.  VAET: A Visual Analytics Approach for E-Transactions Time-Series , 2014, IEEE Transactions on Visualization and Computer Graphics.

[28]  Wei Chen,et al.  ViDX: Visual Diagnostics of Assembly Line Performance in Smart Factories , 2017, IEEE Transactions on Visualization and Computer Graphics.

[29]  Xiaoling Zhang,et al.  Multiway kernel independent component analysis based on feature samples for batch process monitoring , 2009, Neurocomputing.

[30]  Xing Xie,et al.  Discovering regions of different functions in a city using human mobility and POIs , 2012, KDD.

[31]  Christos Faloutsos,et al.  Fast mining and forecasting of complex time-stamped events , 2012, KDD.

[32]  Yu Liu,et al.  Graph Topic Scan Statistic for Spatial Event Detection , 2016, CIKM.

[33]  Yanchi Liu,et al.  Diagnosing New York city's noises with ubiquitous data , 2014, UbiComp.

[34]  Xuelong Li,et al.  Supervised Tensor Learning , 2005, ICDM.

[35]  Andrzej Cichocki,et al.  Nonnegative Matrix and Tensor Factorization T , 2007 .

[36]  Yingcai Wu,et al.  SocialWave: Visual Analysis of Spatio-temporal Diffusion of Information on Social Media , 2017, ACM Trans. Intell. Syst. Technol..

[37]  Yang Bai,et al.  A Multiway Model for Predicting Earthquake Ground Motion , 2013, 2013 14th ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing.

[38]  Philip S. Yu,et al.  Incremental tensor analysis: Theory and applications , 2008, TKDD.

[39]  Hadi Fanaee-T,et al.  Tensor-based anomaly detection: An interdisciplinary survey , 2016, Knowl. Based Syst..

[40]  Mengchen Liu,et al.  A survey on information visualization: recent advances and challenges , 2014, The Visual Computer.

[41]  Oliver Obst,et al.  Spatiotemporal Anomaly Detection in Gas Monitoring Sensor Networks , 2008, EWSN.

[42]  Yingcai Wu,et al.  Embedding Spatio-Temporal Information into Maps by Route-Zooming , 2017, IEEE Transactions on Visualization and Computer Graphics.

[43]  João Gama,et al.  Event detection from traffic tensors: A hybrid model , 2016, Neurocomputing.

[44]  Ciro Cattuto,et al.  Detecting the Community Structure and Activity Patterns of Temporal Networks: A Non-Negative Tensor Factorization Approach , 2013, PloS one.

[45]  Jyrki Kullaa,et al.  Three-way analysis of Structural Health Monitoring data , 2010 .

[46]  Tamara G. Kolda,et al.  Tensor Decompositions and Applications , 2009, SIAM Rev..

[47]  Christos Faloutsos,et al.  MalSpot: Multi2 Malicious Network Behavior Patterns Analysis , 2014, PAKDD.

[48]  Qi Yu,et al.  Fast Multivariate Spatio-temporal Analysis via Low Rank Tensor Learning , 2014, NIPS.

[49]  Wei Chen,et al.  A survey of network anomaly visualization , 2017, Science China Information Sciences.

[50]  Aristides Gionis,et al.  Event detection in activity networks , 2014, KDD.

[51]  G. Smaragdakis,et al.  Spatio-Temporal Network Anomaly Detection by Assessing Deviations of Empirical Measures , 2009, IEEE/ACM Transactions on Networking.