LSOF: Novel Outlier Detection Approach Based on Local Structure

Many local outlier detection algorithms have been proposed inspired by the idea of local outlier factor (LOF). However, they often have low detection performance and are sensitive to neighborhood size because there is a major defect in their calculation formulas of outlier degree and the kNN (k-nearest neighbors) method is widely used to quantify a neighborhood of an instance. To address these issues, we define a novel nearest neighbors tree (NNT) to measure a neighborhood of an instance. Meanwhile, we propose a local structure outlier factor (LSOF), which score each local structure instead of each data point and report the top-scored local structures as anomalous local structures, where outliers and groups of outliers are easily divided according to characteristics of the NNT. Our experimental results demonstrate that the competitive behavior of our method on both synthetic and real-world datasets.

[1]  Pasi Fränti,et al.  Outlier Detection Using k-Nearest Neighbour Graph , 2004, ICPR.

[2]  Osmar R. Zaïane,et al.  A Nonparametric Outlier Detection for Effectively Discovering Top-N Outliers from Engineering Data , 2006, PAKDD.

[3]  Ching-Yung Lin,et al.  A Survey on Social Media Anomaly Detection , 2016, SIGKDD Explor..

[4]  Alexandros Nanopoulos,et al.  Reverse Nearest Neighbors in Unsupervised Distance-Based Outlier Detection , 2015, IEEE Transactions on Knowledge and Data Engineering.

[5]  Valentino Constantinou,et al.  Detecting Spacecraft Anomalies Using LSTMs and Nonparametric Dynamic Thresholding , 2018, KDD.

[6]  Sukumar Nandi,et al.  NDoT: Nearest Neighbor Distance Based Outlier Detection Technique , 2011, PReMI.

[7]  Charu C. Aggarwal,et al.  An Introduction to Outlier Analysis , 2013 .

[8]  Gabriel Maciá-Fernández,et al.  Anomaly-based network intrusion detection: Techniques, systems and challenges , 2009, Comput. Secur..

[9]  Ji Feng,et al.  Natural neighbor: A self-adaptive neighborhood method without parameter K , 2016, Pattern Recognit. Lett..

[10]  A. F. Adams,et al.  The Survey , 2021, Dyslexia in Higher Education.

[11]  Aleksandar Lazarevic,et al.  Outlier Detection with Kernel Density Functions , 2007, MLDM.

[12]  Anthony K. H. Tung,et al.  Ranking Outliers Using Symmetric Neighborhood Relationship , 2006, PAKDD.

[13]  Hans-Peter Kriegel,et al.  LOF: identifying density-based local outliers , 2000, SIGMOD '00.

[14]  Divesh Srivastava,et al.  Reverse Nearest Neighbor Aggregates Over Data Streams , 2002, VLDB.

[15]  S. Muthukrishnan,et al.  Influence sets based on reverse nearest neighbor queries , 2000, SIGMOD '00.

[16]  Arjan Durresi,et al.  A survey: Control plane scalability issues and approaches in Software-Defined Networking (SDN) , 2017, Comput. Networks.

[17]  Arthur Zimek,et al.  On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study , 2016, Data Mining and Knowledge Discovery.

[18]  Arthur Zimek,et al.  Outlier Detection in Urban Traffic Data , 2018, WIMS.

[19]  Rose Yu,et al.  GLAD: group anomaly detection in social media analysis , 2014, ACM Trans. Knowl. Discov. Data.

[20]  Jian Tang,et al.  Enhancing Effectiveness of Outlier Detections for Low Density Patterns , 2002, PAKDD.

[21]  Haibo He,et al.  A local density-based approach for outlier detection , 2017, Neurocomputing.

[22]  Ji Feng,et al.  A non-parameter outlier detection algorithm based on Natural Neighbor , 2016, Knowl. Based Syst..

[23]  Md. Rafiqul Islam,et al.  A survey of anomaly detection techniques in financial domain , 2016, Future Gener. Comput. Syst..

[24]  J. Charles,et al.  A Sino-German λ 6 cm polarization survey of the Galactic plane I . Survey strategy and results for the first survey region , 2006 .

[25]  Osmar R. Zaïane,et al.  Resolution-based outlier factor: detecting the top-n most outlying data points in engineering data , 2008, Knowledge and Information Systems.

[26]  N. N. R. Ranga Suri,et al.  Research Issues in Outlier Detection , 2019, Intelligent Systems Reference Library.