论文信息 - Anomaly Detection and Characterization in Spatial Time Series Data: A Cluster-Centric Approach

2003 - IEEE Trans. Knowl. Data Eng.

A New Temporal Pattern Identification Method for Characterization and Prediction of Complex Time Series Events

A new method for analyzing time series data is introduced in this paper. Inspired by data mining, the new method employs time-delayed embedding and identifies temporal patterns in the resulting phase spaces. An optimization method is applied to search the phase spaces for optimal heterogeneous temporal pattern clusters that reveal hidden temporal patterns, which are characteristic and predictive of time series events. The fundamental concepts and framework of the method are explained in detail. The method is then applied to the characterization and prediction, with a high degree of accuracy, of the release of metal droplets from a welder. The results of the method are compared to those from a Time Delay Neural Network and the C4.5 decision tree algorithm.

2002 - Optics + Photonics

Genetic Algorithms and Support Vector Machines for Time Series Classification

We introduce an algorithm for classifying time series data. Since our initial application is for lightning data, we call the algorithm Zeus. Zeus is a hybrid algorithm that employs evolutionary computation for feature extraction, and a support vector machine for the final backend classification. Support vector machines have a reputation for classifying in high-dimensional spaces without overfitting, so the utility of reducing dimensionality with an intermediate feature selection step has been questioned. We address this question by testing Zeus on a lightning classification task using data acquired from the Fast On-orbit Recording of Transient Events (FORTE) satellite.

1967 - Management Science

The Analysis of Simulation-Generated Time Series

This study applies spectral analysis to the study of time series data generated by simulated stochastic models. Because these data are autocorrelated, analysis by methods applicable to independent observations is not possible. Mathematical models known as covariance stationary stochastic processes are useful representations of autocorrelated time series. The increased publication of literature describing stochastic processes and spectral analysis, in particular, is making these ideas available to an increasing audience. Section I presents a rationale for our interest in time series models and spectral analysis. Section II describes the basic notions of covariance stationary processes. It emphasizes the equivalence of these processes in both the time and frequency domains; the compactness of frequency domain analysis seemingly recommends it over correlation analysis. Section III provides a heuristic background for understanding statistical spectral analysis. Simple frequency-domain statistical properties are emphasized and compared with the rather involved sampling properties of estimated correlograms. Several relevant statistical tests are described. Three simulated experiments are used as examples of how to apply spectral analysis. These are described in Section IV. They are (1) a single-server, first-come-first-served (FCFS) queueing problem with Poisson arrivals and exponentially distributed service times; (2) a similar model with the FCFS assignment rule replaced by one that chooses the job with the shortest service time; and (3) yet another similar model, but with constant service time. In this third example both assignment rules are equivalent. The state of the queue was observed and recorded at unit intervals for all three examples and forms the time series on which Section V is based; theoretical and statistical correlograms and spectra are compared for example 1; examples 1 and 2 are compared using estimated correlograms and spectra; example 3 serves to show some unusual properties of spectra. A particular conclusion of this paper is that differences in the statistical properties of queue length that result under various assumptions and operating rules are easily identified using spectral analysis. More generally, however, this estimation procedure provides a tool for (1) comparing simulated time series with real-world data, and (2) for understanding the implications that various alternative assumptions have on the output of simulated stochastic models.

2011 - International Journal of Forecasting

Advances in forecasting with neural networks? Empirical evidence from the NN3 competition on time series prediction

This paper reports the results of the NN3 competition, which is a replication of the M3 competition with an extension of the competition towards neural network (NN) and computational intelligence (CI) methods, in order to assess what progress has been made in the 10 years since the M3 competition. Two masked subsets of the M3 monthly industry data, containing 111 and 11 empirical time series respectively, were chosen, controlling for multiple data conditions of time series length (short/long), data patterns (seasonal/non-seasonal) and forecasting horizons (short/medium/long). The relative forecasting accuracy was assessed using the metrics from the M3, together with later extensions of scaled measures, and non-parametric statistical tests. The NN3 competition attracted 59 submissions from NN, CI and statistics, making it the largest CI competition on time series data. Its main findings include: (a) only one NN outperformed the damped trend using the sMAPE, but more contenders outperformed the AutomatANN of the M3; (b) ensembles of CI approaches performed very well, better than combinations of statistical methods; (c) a novel, complex statistical method outperformed all statistical and CI benchmarks; and (d) for the most difficult subset of short and seasonal series, a methodology employing echo state neural networks outperformed all others. The NN3 results highlight the ability of NN to handle complex data, including short and seasonal time series, beyond prior expectations, and thus identify multiple avenues for future research.

2012 - ICCSA

Multivariate Time Series Classification by Combining Trend-Based and Value-Based Approximations

Multivariate time series data often have a very high dimensionality. Classifying such high dimensional data poses a challenge because a vast number of features can be extracted. Furthermore, the meaning of the normally intuitive term "similar to" needs to be precisely defined. Representing the time series data effectively is an essential task for decision-making activities such as prediction, clustering and classification. In this paper we propose a feature-based classification approach to classify real-world multivariate time series generated by drilling rig sensors in the oil and gas industry. Our approach encompasses two main phases: representation and classification. For the representation phase, we propose a novel representation of time series which combines trend-based and value-based approximations (we abbreviate it as TVA). It produces a compact representation of the time series which consists of symbolic strings that represent the trends and the values of each variable in the series. The TVA representation improves both the accuracy and the running time of the classification process by extracting a set of informative features suitable for common classifiers. For the classification phase, we propose a memory-based classifier which takes into account the antecedent results of the classification process. The inputs of the proposed classifier are the TVA features computed from the current segment, as well as the predicted class of the previous segment. Our experimental results on real-world multivariate time series show that our approach enables highly accurate and fast classification of multivariate time series.

2016 - Machine Learning

Expected similarity estimation for large-scale batch and streaming anomaly detection

We present a novel algorithm for anomaly detection on very large datasets and data streams. The method, named EXPected Similarity Estimation (expose), is kernel-based and able to efficiently compute the similarity between new data points and the distribution of regular data. The estimator is formulated as an inner product with a reproducing kernel Hilbert space embedding and makes no assumption about the type or shape of the underlying data distribution. We show that offline (batch) learning with exposecan be done in linear time and online (incremental) learning takes constant time per instance and model update. Furthermore, exposecan make predictions in constant time, while it requires only constant memory. In addition, we propose different methodologies for concept drift adaptation on evolving data streams. On several real datasets we demonstrate that our approach can compete with state of the art algorithms for anomaly detection while being an order of magnitude faster than most other approaches.

2012 - Pattern Recognit.

Adaptive ROC-based ensembles of HMMs applied to anomaly detection

Hidden Markov models (HMMs) have been successfully applied in many intrusion detection applications, including anomaly detection from sequences of operating system calls. In practice, anomaly detection systems (ADSs) based on HMMs typically generate false alarms because they are designed using limited amount of representative training data. Since new data may become available over time, an important feature of an ADS is the ability to accommodate newly acquired data incrementally, after it has originally been trained and deployed for operations. In this paper, a system based on the receiver operating characteristic (ROC) is proposed to efficiently adapt ensembles of HMMs (EoHMMs) in response to new data, according to a learn-and-combine approach. When a new block of training data becomes available, a pool of base HMMs is generated from the data using a different number of HMM states and random initializations. The responses from the newly trained HMMs are then combined to those of the previously trained HMMs in ROC space using a novel incremental Boolean combination (incrBC) technique. Finally, specialized algorithms for model management allow to select a diversified EoHMM from the pool, and adapt Boolean fusion functions and thresholds for improved performance, while it prunes redundant base HMMs. The proposed system is capable of changing the desired operating point during operations, and this point can be adjusted to changes in prior probabilities and costs of errors. Computer simulations conducted on synthetic and real-world host-based intrusion detection data indicate that the proposed system can achieve a significantly higher level of performance than when parameters of a single best HMM are estimated, at each learning stage, using reference batch and incremental learning techniques. It also outperforms the learn-and-combine approaches using static fusion functions (e.g., majority voting). Over time, the proposed ensemble selection algorithms form compact EoHMMs, while maintaining or improving system accuracy. Pruning allows to limit the pool size from increasing indefinitely, thereby reducing the storage space for accommodating HMMs parameters without negatively affecting the overall EoHMM performance. Although applied for HMM-based ADSs, the proposed approach is general and can be employed for a wide range of classifiers and detection applications.

2017 - 2017 IEEE International Joint Conference on Biometrics (IJCB)

An anomaly detection approach to face spoofing detection: A new formulation and evaluation protocol

Face anti-spoofing problem can be quite challenging due to various factors including diversity of face spoofing attacks, any new means of spoofing, the problem of imaging sensor interoperability and other environmental factors in addition to the small sample size. Taking into account these observations, in this work, first, a new evaluation protocol called “innovative attack evaluation protocol” to study the effect of occurrence of unseen attack types is proposed which better reflects the realistic conditions in spoofing attacks. Second, a new formulation of the problem based on the anomaly detection concept is proposed where the training data comes from the positive class only. The test data, of course, may come from the positive or negative class. Finally, a thorough evaluation and comparison of 20 different one-class and two-class systems is performed and demonstrated that the anomaly-based formulation is not inferior as compared with the conventional two-class approach.

2020 - ArXiv

Deep Transformer Models for Time Series Forecasting: The Influenza Prevalence Case

In this paper, we present a new approach to time series forecasting. Time series data are prevalent in many scientific and engineering disciplines. Time series forecasting is a crucial task in modeling time series data, and is an important area of machine learning. In this work we developed a novel method that employs Transformer-based machine learning models to forecast time series data. This approach works by leveraging self-attention mechanisms to learn complex patterns and dynamics from time series data. Moreover, it is a generic framework and can be applied to univariate and multivariate time series data, as well as time series embeddings. Using influenza-like illness (ILI) forecasting as a case study, we show that the forecasting results produced by our approach are favorably comparable to the state-of-the-art.

2007 - 2007 IEEE International Conference on Image Processing

Fast Hyperspectral Anomaly Detection via SVDD

We present a method for fast anomaly detection in hyperspectral imagery (HSI) based on the support vector data description (SVDD) algorithm. The SVDD is a single class, non-parametric approach for modeling the support of a distribution. A global SVDD anomaly detector is developed that utilizes the SVDD to model the distribution of the spectra of pixels randomly selected from the entire image. Experiments on wide area airborne mine detection (WAAMD) hyperspectral data show improved receiver operating characteristic (ROC) detection performance when compared to the local SVDD detector and other standard anomaly detectors (including RX and GMRF). Furthermore, one-second processing time using desktop computers on several 256 times 256 times 145 datacubes is achieved.

2000 - ANLP

Detecting Errors within a Corpus using Anomaly Detection

We present a method for automatically detecting errors in a manually marked corpus using anomaly detection. Anomaly detection is a method for determining which elements of a large data set do not conform to the whole. This method fits a probability distribution over the data and applies a statistical test to detect anomalous elements. In the corpus error detection problem, anomalous elements are typically marking errors. We present the results of applying this method to the tagged portion of the Penn Treebank corpus.

2007 - ASIACCS '07

Network anomaly detection based on TCM-KNN algorithm

Intrusion detection is a critical component of secure information systems. Network anomaly detection has been an active and difficult research topic in the field of Intrusion Detection for many years. However, it still has some problems unresolved. They include high false alarm rate, difficulties in obtaining exactly clean data for the modeling of normal patterns and the deterioration of detection rate because of some "noisy" data in the training set. In this paper, we propose a novel network anomaly detection method based on improved TCM-KNN (Transductive Confidence Machines for K-Nearest Neighbors) machine learning algorithm. A series of experimental results on the well-known KDD Cup 1999 dataset demonstrate it can effectively detect anomalies with high true positive rate, low false positive rate and high confidence than the state-of-the-art anomaly detection methods. In addition, even interfered by "noisy" data (unclean data), the proposed method is robust and effective. Moreover, it still retains good detection performance after employing feature selection aiming at avoiding the "curse of dimensionality".

2018 - NeurIPS

Multivariate Time Series Imputation with Generative Adversarial Networks

Multivariate time series usually contain a large number of missing values, which hinders the application of advanced analysis methods on multivariate time series data. Conventional approaches to addressing the challenge of missing values, including mean/zero imputation, case deletion, and matrix factorization-based imputation, are all incapable of modeling the temporal dependencies and the nature of complex distribution in multivariate time series. In this paper, we treat the problem of missing value imputation as data generation. Inspired by the success of Generative Adversarial Networks (GAN) in image generation, we propose to learn the overall distribution of a multivariate time series dataset with GAN, which is further used to generate the missing values for each sample. Different from the image data, the time series data are usually incomplete due to the nature of data recording process. A modified Gate Recurrent Unit is employed in GAN to model the temporal irregularity of the incomplete time series. Experiments on two multivariate time series datasets show that the proposed model outperformed the baselines in terms of accuracy of imputation. Experimental results also showed that a simple model on the imputed data can achieve state-of-the-art results on the prediction tasks, demonstrating the benefits of our model in downstream applications.

2014 - Int. J. Approx. Reason.

Anomaly detection in vessel tracks using Bayesian networks

Abstract In recent years electronic tracking has provided voluminous data on vessel movements, leading researchers to try various data mining techniques to find patterns and, especially, deviations from patterns, i.e., for anomaly detection. Here we describe anomaly detection with data mined Bayesian Networks, learning them from real world Automated Identification System (AIS) data, and from supplementary data, producing both dynamic and static Bayesian network models. We find that the learned networks are quite easy to examine and verify despite incorporating a large number of variables. We also demonstrate that combining dynamic and static modelling approaches improves the coverage of the overall model and thereby anomaly detection performance.

2006 - NIPS

In-Network PCA and Anomaly Detection

We consider the problem of network anomaly detection in large distributed systems. In this setting, Principal Component Analysis (PCA) has been proposed as a method for discovering anomalies by continuously tracking the projection of the data onto a residual subspace. This method was shown to work well empirically in highly aggregated networks, that is, those with a limited number of large nodes and at coarse time scales. This approach, however, has scalability limitations. To overcome these limitations, we develop a PCA-based anomaly detector in which adaptive local data filters send to a coordinator just enough data to enable accurate global detection. Our method is based on a stochastic matrix perturbation analysis that characterizes the tradeoff between the accuracy of anomaly detection and the amount of data communicated over the network.

2011 - PPOPP '90

An empirical comparison of monitoring algorithms for access anomaly detection

One of the major disadvantages of parallel programming with shared memory is the nondeterministic behavior caused by uncoordinated access to shared variables, known as <italic>access anomalies</italic>. Monitoring program execution to detect access anomalies is a promising and relatively unexplored approach to this problem. We present a new algorithm, referred to as <italic>task recycling</italic>, for detecting anomalies, and compare it to an existing algorithm. Empirical results indicate several significant conclusions: (i) While space requirements are bounded by &Ogr;(<italic>T</italic> × <italic>V</italic>), where <italic>T</italic> is the maximum number of threads that may potentially execute in parallel and <italic>V</italic> is the number of variable monitored, for typical programs space requirements are on average &Ogr;(<italic>V</italic>). (ii) Task recycling is more efficient in terms of space requirements and often in performance. (iii) The general approach of monitoring to detect access anomalies is practical.

2016 - ICML

Deep Structured Energy Based Models for Anomaly Detection

In this paper, we attack the anomaly detection problem by directly modeling the data distribution with deep architectures. We propose deep structured energy based models (DSEBMs), where the energy function is the output of a deterministic deep neural network with structure. We develop novel model architectures to integrate EBMs with different types of data such as static data, sequential data, and spatial data, and apply appropriate model architectures to adapt to the data structure. Our training algorithm is built upon the recent development of score matching \cite{sm}, which connects an EBM with a regularized autoencoder, eliminating the need for complicated sampling method. Statistically sound decision criterion can be derived for anomaly detection purpose from the perspective of the energy landscape of the data distribution. We investigate two decision criteria for performing anomaly detection: the energy score and the reconstruction error. Extensive empirical studies on benchmark tasks demonstrate that our proposed model consistently matches or outperforms all the competing methods.

2008 - IMC '08

An empirical evaluation of entropy-based traffic anomaly detection

Entropy-based approaches for anomaly detection are appealing since they provide more fine-grained insights than traditional traffic volume analysis. While previous work has demonstrated the benefits of entropy-based anomaly detection, there has been little effort to comprehensively understand the detection power of using entropy-based analysis of multiple traffic distributions in conjunction with each other. We consider two classes of distributions: flow-header features (IP addresses, ports, and flow-sizes), and behavioral features (degree distributions measuring the number of distinct destination/source IPs that each host communicates with). We observe that the timeseries of entropy values of the address and port distributions are strongly correlated with each other and provide very similar anomaly detection capabilities. The behavioral and flow size distributions are less correlated and detect incidents that do not show up as anomalies in the port and address distributions. Further analysis using synthetically generated anomalies also suggests that the port and address distributions have limited utility in detecting scan and bandwidth flood anomalies. Based on our analysis, we discuss important implications for entropy-based anomaly detection.

2016 - Pattern Recognit.

High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning

High-dimensional problem domains pose significant challenges for anomaly detection. The presence of irrelevant features can conceal the presence of anomalies. This problem, known as the 'curse of dimensionality', is an obstacle for many anomaly detection techniques. Building a robust anomaly detection model for use in high-dimensional spaces requires the combination of an unsupervised feature extractor and an anomaly detector. While one-class support vector machines are effective at producing decision surfaces from well-behaved feature vectors, they can be inefficient at modelling the variation in large, high-dimensional datasets. Architectures such as deep belief networks (DBNs) are a promising technique for learning robust features. We present a hybrid model where an unsupervised DBN is trained to extract generic underlying features, and a one-class SVM is trained from the features learned by the DBN. Since a linear kernel can be substituted for nonlinear ones in our hybrid model without loss of accuracy, our model is scalable and computationally efficient. The experimental results show that our proposed model yields comparable anomaly detection performance with a deep autoencoder, while reducing its training and testing time by a factor of 3 and 1000, respectively. HighlightsWe use a combination of a one-class SVM and deep learning.In our model linear kernels can be used rather than nonlinear ones.Our model delivers a comparable accuracy with a deep autoencoder.Our model executes 3times faster in training and 1000 faster than a deep autoencoder.

2014 - IEEE Communications Surveys & Tutorials

Network Anomaly Detection: Methods, Systems and Tools

Network anomaly detection is an important and dynamic research area. Many network intrusion detection methods and systems (NIDS) have been proposed in the literature. In this paper, we provide a structured and comprehensive overview of various facets of network anomaly detection so that a researcher can become quickly familiar with every aspect of network anomaly detection. We present attacks normally encountered by network intrusion detection systems. We categorize existing network anomaly detection methods and systems based on the underlying computational techniques used. Within this framework, we briefly describe and compare a large number of network anomaly detection methods and systems. In addition, we also discuss tools that can be used by network defenders and datasets that researchers in network anomaly detection can use. We also highlight research directions in network anomaly detection.

Anomaly Detection and Characterization in Spatial Time Series Data: A Cluster-Centric Approach

A Pre-clustering Method To Improve Anomaly Detection

Towards Visual Exploration of Large Temporal Datasets

Time Series Data Mining Methods : A Review

Bound smoothing based time series anomaly detection using multiple similarity measures

A Survey on Data Mining Methods for Clustering Complex Spatiotemporal Data

Takagi–Sugeno Fuzzy Modeling Using Mixed Fuzzy Clustering

A fixed-charge transportation problem in two-stage supply chain network in Gaussian type-2 fuzzy environments

A survey on novelty detection using level set methods

A heuristic approach to detect novelty data using improved level set methods

A conceptual paper on novelty detection for temporal data using level set methods

A Self-Learning and Online Algorithm for Time Series Anomaly Detection, with Application in CPU Manufacturing

Exact variable-length anomaly detection algorithm for univariate and multivariate time series

UK - Means Clustering for Uncertain Time Series Based on ULDTW Distance

Entropic One-Class Classifiers

ADARC: An anomaly detection algorithm based on relative outlier distance and biseries correlation

From Rocks to Pebbles

COPE: Interactive Exploration of Co-Occurrence Patterns in Spatial Time Series

A Geometric Approach to Clustering Based Anomaly Detection for Industrial Applications

Anomaly Detection Guidelines for Data Streams in Big Data

Root-cause Analysis for Time-series Anomalies via Spatiotemporal Graphical Modeling in Distributed Complex Systems