Traffic outlier detection by density-based bounded local outlier factors

Outlier detection (OD) is widely used in many fields, such as finance, information and medicine, in cleaning up datasets and keeping the useful information. In a traffic system, it alerts the transport department and drivers with abnormal traffic situations such as congestion and traffic accident. This paper presents a density-based bounded LOF (BLOF) method for large-scale traffic video data in Hong Kong. A dimension reduction by principal component analysis (PCA) was accomplished on the spatial-temporal traffic signals. Previously, a density-based local outlier factor (LOF) method on a two-dimensional (2D) PCA-proceeded spatial plane was performed. In this paper, a three-dimensional (3D) PCA-proceeded spatial space for the classical density-based OD is firstly compared with the results from the 2D counterpart. In our experiments, the classical density-based LOF OD has been applied to the 3D PCA-proceeded data domain, which is new in literature, and compared to the previous 2D domain. The average DSRs has increased about 2% in the PM sessions: 91% (2D) and 93% (3D). Also, comparing the classical density-based LOF and the new BLOF OD methods, the average DSRs in the supervised approach has increased from 94% (LOF) to 96% (BLOF) for the AM sessions and from 93% (LOF) to 95% (BLOF) for the PM sessions.

[1]  Wei Liu,et al.  Distance-based k-nearest neighbors outlier detection method in large-scale traffic data , 2015, 2015 IEEE International Conference on Digital Signal Processing (DSP).

[2]  Slim Abdennadher,et al.  Enhancing one-class support vector machines for unsupervised anomaly detection , 2013, ODD '13.

[3]  Prabhakar Raghavan,et al.  A Linear Method for Deviation Detection in Large Databases , 1996, KDD.

[4]  Junshui Ma,et al.  Online novelty detection on temporal sequences , 2003, KDD '03.

[5]  Muhammad Ali Imran,et al.  Anomaly Detection in Wireless Sensor Networks in a Non-Stationary Environment , 2014, IEEE Communications Surveys & Tutorials.

[6]  Aleksandar Lazarevic,et al.  Outlier Detection with Kernel Density Functions , 2007, MLDM.

[7]  Wei Wang,et al.  A comparison of outlier detection algorithms for ITS data , 2010, Expert Syst. Appl..

[8]  Amruta D. Pawar,et al.  A Survey on Outlier Detection Techniques for Credit Card Fraud Detection , 2014 .

[9]  Alberto Cardoso,et al.  Dealing With Outliers in Wireless Sensor Networks: An Oil Refinery Application , 2014, IEEE Transactions on Control Systems Technology.

[10]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[11]  Bijaya K. Panigrahi,et al.  Prediction interval estimation for wind farm power generation forecasts using support vector machines , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[12]  Henry Y. T. Ngan,et al.  Robust defect detection in plain and twill fabric using directional Bollinger bands , 2015 .

[13]  Klemens Böhm,et al.  Ranking outlier nodes in subspaces of attributed graphs , 2013, 2013 IEEE 29th International Conference on Data Engineering Workshops (ICDEW).

[14]  Grantham Pang,et al.  Fabric inspection based on the Elo rating method , 2016, Pattern Recognit..

[15]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[16]  Eyal Amir,et al.  Real-time Bayesian Anomaly Detection for Environmental Sensor Data , 2007 .

[17]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[18]  Peter J. Rousseeuw,et al.  Robust regression and outlier detection , 1987 .

[19]  Raymond T. Ng,et al.  Distance-based outliers: algorithms and applications , 2000, The VLDB Journal.

[20]  Mei Bai,et al.  Density-Based Local Outlier Detection on Uncertain Data , 2014, WAIM.

[21]  Wei Liu,et al.  Density-based Outlier Detection by Local Outlier Factor on Largescale Traffic Data , 2016 .

[22]  Michael K. Ng,et al.  Patterned Fabric Inspection and Visualization by the Method of Image Decomposition , 2014, IEEE Transactions on Automation Science and Engineering.

[23]  Nelson H. C. Yung,et al.  A comparative study of outlier detection for large-scale traffic data by one-class SVM and kernel density estimation , 2015, Electronic Imaging.

[24]  Fei Tony Liu,et al.  Isolation-Based Anomaly Detection , 2012, TKDD.

[25]  Robert P. W. Duin,et al.  Outlier Detection Using Classifier Instability , 1998, SSPR/SPR.

[26]  Simon J. Godsill,et al.  Detection of abrupt spectral changes using support vector machines an application to audio signal segmentation , 2002, 2002 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[27]  Nelson H. C. Yung,et al.  Outlier Detection in Traffic Data Based on the Dirichlet Process Mixture Model , 2015 .