With the widespread usage of 4G technologies and the upcoming promise of 5G networks, there is a strong need for increased network performance and reliability. However, as these networks become bigger and faster, so does their complexity. Currently, network operators detect most of the network failures manually. This is a very time consuming and tedious task for them, oftentimes taking up to several hours. Thereby arises a need for an automated Anomaly Detection and Correction system. Such a system would be a step towards the ultimate goal of a cognitive self-organizing network. We here take the case of a mobile network with hundreds of key performance indicators, which generates huge amount of network logs every hour. Since user behavior has patterns in usage, e.g. weekdays network traffic will be higher than weekend’s traffic near office areas, we analyze a Time Series (TS) Decomposition based approach, which takes into consideration of trends and seasonality in data. We also explore the use of a seasonal auto-regressive technique, SARIMA, for anomaly detection. Assuming that an anomalous behaviour is continuous in time, we evaluate a recurrent encoder-decoder based approach, MSCRED for Anomalous Window Detection. We do this analysis to find the KPI and the respective network element, whose behavior is abnormal. Our results show that while Time Series Decomposition outperforms SARIMA over single point anomaly detection, MSCRED significantly performs well in predicting anomalous time windows.

With the development of the Cyber-Physical-Social Systems(CPSS), a large number of multidimensional time series have been generated in today’s world, such as: sensor data for industrial equipment operation, vehicle driving data, and cloud server operation and maintenance data and so on. The key task of Cloud–Fog–Edge Computing in managing these systems is how to detect anomalous data in a specific time series to facilitate operator action to solve potential system problems. So multidimensional time series outlier detection become an important direction of CPSS data mining and Cloud–Fog–Edge Computing research, and it has a wide range of applications in industry, finance, medicine and other fields. This paper proposes a framework called Multidimensional time series Outlier detection based on a Time Convolutional Network AutoEncoder (MOTCN-AE), which can detect outliers in time series data, such as identifying equipment failures, dangerous driving behaviors of cars, etc. Specifically, this paper first uses a feature extraction method to transform the original time series into a feature-rich time series. Second, the proposed TCN-AE is used to reconstruct the feature-rich time series data, and the reconstruction error is used to calculate outlier scores. Finally, the MOTCN-AE framework is validated by multiple time series datasets to demonstrate its effectiveness in detecting time series outliers.

Wind power, as an alternative to burning fossil fuels, is plentiful and renewable. Data-driven approaches are increasingly popular for inspecting the wind turbine failures. In this paper, we propose a novel classification-based anomaly detection system for icing detection of the wind turbine blades. We effectively combine the deep neural networks and wavelet transformation to identify such failures sequentially across the time. In the training phase, we present a wavelet based fully convolutional neural network (FCNN), namely WaveletFCNN, for the time series classification. We improve the original (FCNN) by augmenting features with the wavelet coefficients. WaveletFCNN outperforms the state-of-the-art FCNN for the univariate time series classification on the UCR time series archive benchmarks. In the detecting phase, we combine the sliding window and majority vote algorithms to provide the timely monitoring of the anomalies. The system has been successfully implemented on a real-world dataset from Goldwind Inc, where the classifier is trained on a multivariate time series dataset and the monitoring algorithm is implemented to capture the abnormal condition on signals from a wind farm.

Wind power, as an alternative to burning fossil fuels, is abundant and inexhaustible. To fully utilize wind power, wind farms are usually located in areas of high altitude and facing serious ice conditions, which can lead to serious consequences. Quick detection of blade ice accretion is crucial for the maintenance of wind farms. Unlike traditional methods of installing expensive physical detectors on wind blades, data-driven approaches are increasingly popular for inspecting the wind turbine failures. In this work, we propose a wavelet enhanced autoencoder model (WaveletAE) to identify wind turbine dysfunction by analyzing the multivariate time series monitored by the SCADA system. WaveletAE is enhanced with wavelet detail coefficients to enforce the autoencoder to capture information from multiple scales, and the CNN-LSTM architecture is applied to learn channel-wise and temporal-wise relations. The empirical study shows that the proposed model outperforms other state-of-the-art time series anomaly detection methods for real-world blade icing detection.

We propose the application of unsupervised machine learning to automatically detect anomalous behavior preceding machine failure. We achieve this through an approach that utilizes Principal Component Analysis (PCA), latent feature extraction, and Density Based Scanning of Applications with Noise (DBSCAN). We call this method AnomDB. Time series data collected from Computer Numerically Controlled (CNC) machines may benefit from this technique due to its ability to consolidate noisy, multivariate data from CNC controls and detect anomalies without reliance on periodicity of signal. We perform experiments on CNC machine control data to demonstrate the effectiveness of this method in discovering anomalies over other commonly used methods of anomaly detection such as IQR and k-means clustering. We show the effectiveness of this method on a representative example in an actual machine shop, and then on a series of real machining data with synthetic anomalies injected.

We present a novel framework for multivariate time series representation learning based on the transformer encoder architecture. The framework includes an unsupervised pre-training scheme, which can offer substantial performance benefits over fully supervised learning on downstream tasks, both with but even without leveraging additional unlabeled data, i.e., by reusing the existing data samples. Evaluating our framework on several public multivariate time series datasets from various domains and with diverse characteristics, we demonstrate that it performs significantly better than the best currently available methods for regression and classification, even for datasets which consist of only a few hundred training samples. Given the pronounced interest in unsupervised learning for nearly all domains in the sciences and in industry, these findings represent an important landmark, presenting the first unsupervised method shown to push the limits of state-of-the-art performance for multivariate time series regression and classification.

Time series prediction has been challenging topic in several application domains. In this paper, an ensemble of two top performing deep learning architectures across different applications such as fresh produce (FP) yield prediction, FP price prediction and crude oil price prediction is proposed. First, the input data is trained on an array of different machine learning architectures, the top two performers are then combined using a stacking ensemble. The top two performers across the three tested applications are found to be Attention CNN-LSTM (AC-LSTM) and Attention ConvLSTM (ACV-LSTM). Different ensemble techniques, mean prediction, Linear Regression (LR) and Support vector Regression (SVR), are then utilized to come up with the best prediction. An aggregated measure that combines the results of mean absolute error (MAE), mean squared error (MSE) and R2 coefficient of determination (R2) is used to evaluate model performance. The experiment results show that across the various examined applications, the proposed model which is a stacking ensemble of the AC-LSTM and ACV-LSTM using a linear SVR is the best performing based on the aggregated measure.

The monitoring and management of numerous and diverse time series data at Alibaba Group calls for an effective and scalable time series anomaly detection service. In this paper, we propose RobustTAD, a Robust Time series Anomaly Detection framework by integrating robust seasonal-trend decomposition and convolutional neural network for time series data. The seasonal-trend decomposition can effectively handle complicated patterns in time series, and meanwhile significantly simplifies the architecture of the neural network, which is an encoder-decoder architecture with skip connections. This architecture can effectively capture the multi-scale information from time series, which is very useful in anomaly detection. Due to the limited labeled data in time series anomaly detection, we systematically investigate data augmentation methods in both time and frequency domains. We also introduce label-based weight and value-based weight in the loss function by utilizing the unbalanced nature of the time series anomaly detection problem. Compared with the widely used forecasting-based anomaly detection algorithms, decomposition-based algorithms, traditional statistical algorithms, as well as recent neural network based algorithms, RobustTAD performs significantly better on public benchmark datasets. It is deployed as a public online service and widely adopted in different business scenarios at Alibaba Group.

The increasing accessibility of data provides substantial opportunities for understanding user behaviors. Unearthing anomalies in user behaviors is of particular importance as it helps signal harmful incidents such as network intrusions, terrorist activities, and financial frauds. Many visual analytics methods have been proposed to help understand user behavior-related data in various application domains. In this work, we survey the state of art in visual analytics of anomalous user behaviors and classify them into four categories including social interaction, travel, network communication, and transaction. We further examine the research works in each category in terms of data types, anomaly detection techniques, and visualization techniques, and interaction methods. Finally, we discuss the findings and potential research directions.

The importance of imputation for having highly performing prediction models is highlighted in this work. Three imputation techniques are tested against a non-imputation approach that discards records with any missing values; the complete-case analysis (CCA). The deep learning linear memory vector recurrent neural network-RNN (LIME) imputation model is tested along with two other nondeep learning models such as the linear function and Last Observation Carried Forward (LOCF). The simple LSTM deep learning (DL) prediction model is deployed to decide the best performing imputation model, the one resulting in the lowest price and yield prediction errors. Five performance evaluation measures are utilized; the mean absolute error (MAE), the root mean square error (RMSE), R2 correlation measure along with two aggregated measures summarizing these three measures to decide the overall prediction performance; the average aggregated measure (AGM) for each considered step ahead and the average of the AGM across all considered steps ahead (AAGM). Based on AGM, it is found that the LIME imputation model leads to the best prediction performance of the simple LSTM DL model across both applications of 5 weeks ahead strawberry price and yield predictions using weather; W2P and W2Y. Therefore, the LIME imputed file is reused to train two compound DL models, Convolutional Long Short-Term Memory RNN with attention (ATT-ConvLSTM) and ATT-CNN-LSTM along with their Voting Regressor ensemble (VR). The same models are retrained with files preprocessed with the non-imputation approach, CCA. It is found that the overall AAGM of the compound DL and ensemble prediction models across all the 1, 2, 3, and 4 weeks ahead price predictions confirm that using LIME highly improves the prediction performance of the ensemble and its compound DL components. The VR ensemble price prediction performance is improved by 72% and the ATTConvLSTM component is improved by 89% compared to their performances without imputation; using CCA preprocessed files.

The challenges surrounding the optimal operation of power systems are growing in various dimensions, due in part to increasingly distributed energy resources and a progression towards large-scale transportation electrification. Currently, the increasing uncertainties associated with both renewable energy generation and demand are largely being managed by increasing operational reserves—potentially at the cost of suboptimal economic conditions—in order to maintain the reliability of the system. This chapter looks at the big picture role of forecasting in power systems from generation to consumption and provides a comprehensive review of traditional approaches for forecasting generation and load in various contexts. This chapter then takes a deep dive into the state-of-the-art machine learning and deep learning approaches for power systems forecasting. Furthermore, a case study of multi-time-horizon solar irradiance forecasting using deep learning is discussed in detail. Smart grids form the backbone of the future interdependent networks. For addressing the challenges associated with the operations of smart grid, development and wide adoption of machine learning and deep learning algorithms capable of producing better forecasting accuracies is urgently needed. Along with exploring the implementation and benefits of these approaches, this chapter also considers the strengths and limitations of deep learning algorithms for power systems forecasting applications. This chapter, thus, provides a panoramic view of state-of-the-art of predictive analytics in power systems in the context of future smart grid operations.

The automatic supervision of IT systems is a current challenge at Orange. Given the size and complexity reached by its IT operations, the number of sensors needed to obtain measurements over time, used to infer normal and abnormal behaviors, has increased dramatically making traditional expert-based supervision methods slow or prone to errors. In this paper, we propose a fast and stable method called UnSupervised Anomaly Detection for multivariate time series (USAD) based on adversely trained autoencoders. Its autoencoder architecture makes it capable of learning in an unsupervised way. The use of adversarial training and its architecture allows it to isolate anomalies while providing fast training. We study the properties of our methods through experiments on five public datasets, thus demonstrating its robustness, training speed and high anomaly detection performance. Through a feasibility study using Orange's proprietary data we have been able to validate Orange's requirements on scalability, stability, robustness, training speed and high performance.

The ability to accurately and consistently discover anomalies in time series is important in many applications. Fields such as finance (fraud detection), information security (intrusion detection), healthcare, and others all benefit from anomaly detection. Intuitively, anomalies in time series are time points or sequences of time points that deviate from normal behavior characterized by periodic oscillations and long-term trends. For example, the typical activity on e-commerce websites exhibits weekly periodicity and grows steadily before holidays. Similarly, domestic usage of electricity exhibits daily and weekly oscillations combined with long-term season-dependent trends. How can we accurately detect anomalies in such domains while simultaneously learning a model for normal behavior? We propose a robust offline unsupervised framework for anomaly detection in seasonal multivariate time series, called AURORA. A key innovation in our framework is a general background behavior model that unifies periodicity and long-term trends. To this end, we leverage a Ramanujan periodic dictionary and a spline-based dictionary to capture both seasonal and trend patterns. We conduct experiments on both synthetic and real-world datasets and demonstrate the effectiveness of our method. AURORA has significant advantages over existing models for anomaly detection, including high accuracy (AUC of up to 0.98), interpretability of recovered normal behavior (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$100\%$$\end{document}100% accuracy in period detection), and the ability to detect both point and contextual anomalies. In addition, AURORA is orders of magnitude faster than baselines.

The Internet-of-Things (IoT) has great potential to fundamentally alter the delivery of modern healthcare, enabling healthcare solutions outside the limits of conventional clinical settings. It can offer ubiquitous monitoring to at-risk population groups and allow diagnostic care, preventive care, and early intervention in everyday life. These services can have profound impacts on many aspects of health and well-being. However, this field is still at an infancy stage, and the use of IoT-based systems in real-world healthcare applications introduces new challenges. Healthcare applications necessitate satisfactory quality attributes such as reliability and accuracy due to their mission-critical nature, while at the same time, IoT-based systems mostly operate over constrained shared sensing, communication, and computing resources. There is a need to investigate this synergy between the IoT technologies and healthcare applications from a user-centered perspective. Such a study should examine the role and requirements of IoT-based systems in realworld health monitoring applications. Moreover, conventional computing architecture and data analytic approaches introduced for IoT systems are insufficient when used to target health and well-being purposes, as they are unable to overcome the limitations of IoT systems while fulfilling the needs of healthcare applications. This thesis aims to address these issues by proposing an intelligent use of data and computing resources in IoT-based systems, which can lead to a high-level performance and satisfy the stringent requirements. For this purpose, this thesis first delves into the state-of-the-art IoTenabled healthcare systems proposed for in-home and in-hospital monitoring. The findings are analyzed and categorized into different domains from a user-centered perspective. The selection of home-based applications is focused on the monitoring of the elderly who require more remote care and support compared to other groups of people. In contrast, the hospital-based applications include the role of existing IoT in patient monitoring and hospital management systems. Then, the objectives and requirements of each domain are investigated and discussed. This thesis proposes personalized data analytic approaches to fulfill the

Temporal anomaly detection looks for irregularities over space-time. Unsupervised temporal models employed thus far typically work on sequences of feature vectors, and much less on temporal multiway data. We focus our investigation on twoway data, in which a data matrix is observed at each time step. Leveraging recent advances in matrix-native recurrent neural networks, we investigated strategies for data arrangement and unsupervised training for temporal multiway anomaly detection. These include compressing-decompressing, encoding-predicting, and temporal data differencing. We conducted a comprehensive suite of experiments to evaluate model behaviors under various settings on synthetic data, moving digits, and ECG recordings. We found interesting phenomena not previously reported. These include the capacity of the compact matrix LSTM to compress noisy data near perfectly, making the strategy of compressingdecompressing data ill-suited for anomaly detection under the noise. Also, long sequence of vectors can be addressed directly by matrix models that allow very long context and multiple step prediction. Overall, the encoding-predicting strategy works very well for the matrix LSTMs in the conducted experiments, thanks to its compactness and better fit to the data dynamics.

Supernovae mark the explosive deaths of stars and enrich the cosmos with heavy elements. Future telescopes will discover thousands of new supernovae nightly, creating a need to flag astrophysically interesting events rapidly for followup study. Ideally, such an anomaly detection pipeline would be independent of our current knowledge and be sensitive to unexpected phenomena. Here we present an unsupervised method to search for anomalous time series in real time for transient, multivariate, and aperiodic signals. We use a RNN-based variational autoencoder to encode supernova time series and an isolation forest to search for anomalous events in the learned encoded space. We apply this method to a simulated dataset of 12,159 supernovae, successfully discovering anomalous supernovae and objects with catastrophically incorrect redshift measurements. This work is the first anomaly detection pipeline for supernovae which works with online datastreams.

Spatial event forecasting is challenging and crucial for urban sensing scenarios, which is beneficial for a wide spectrum of spatial-temporal mining applications, ranging from traffic management, public safety, to environment policy making. In spite of significant progress has been made to solve spatial-temporal prediction problem, most existing deep learning based methods based on a coarse-grained spatial setting and the success of such methods largely relies on data sufficiency. In many real-world applications, predicting events with a fine-grained spatial resolution do play a critical role to provide high discernibility of spatial-temporal data distributions. However, in such cases, applying existing methods will result in weak performance since they may not well capture the quality spatial-temporal representations when training triple instances are highly imbalanced across locations and time. To tackle this challenge, we develop a hierarchically structured Spatial-Temporal ransformer network (STtrans) which leverages a main embedding space to capture the inter-dependencies across time and space for alleviating the data imbalance issue. In our STtrans framework, the first-stage transformer module discriminates different types of region and time-wise relations. To make the latent spatial-temporal representations be reflective of the relational structure between categories, we further develop a cross-category fusion transformer network to endow STtrans with the capability to preserve the semantic signals in a fully dynamic manner. Finally, an adversarial training strategy is introduced to yield a robust spatial-temporal learning under data imbalance. Extensive experiments on real-world imbalanced spatial-temporal datasets from NYC and Chicago demonstrate the superiority of our method over various state-of-the-art baselines.

Social Network Analysis has become a core aspect of analyzing networks today. As statistics blended with computer science gave rise to data mining in machine learning, so is the social network analysis, which finds its roots from sociology and graphs in mathematics. In the past decades, researchers in sociology and social sciences used the data from surveys and employed graph theoretical concepts to study the patterns in the underlying networks. Nowadays, with the growth of technology following Moore’s Law, we have an incredible amount of information generating per day. Most of which is a result of an interplay between individuals, entities, sensors, genes, neurons, documents, etc., or their combinations. With the emerging line of networks such as IoT, Web 2.0, Industry 4.0, smart cities and so on, the data growth is expected to be more aggressive. Analyzing and mining such rapidly generating evolving forms of networks is a real challenge. There are quite a number of research works concentrating on analytics for static and aggregated networks. Nevertheless, as the data is growing faster than computational power, those methods suffer from a number of shortcomings including constraints of space, computation and stale results. While focusing on the above challenges, this dissertation encapsulates contributions in three major perspectives: Analysis, Sampling, and Mining of streaming networks. Stream processing is an exemplary way of treating continuously emerging temporal data. Therefore, in this dissertation, we propose algorithms that comply with single-pass and limited memory for processing. Additionally, to deal with the situations where data generation speed is higher than the processing speed, we present dynamic sampling on evolving networks. Dynamic sampling in streaming scenarios is capable of efficiently managing in-memory data for high-speed networks; This makes it a powerful means to serve many problems such as performing analytics, maintaining sufficient statistics, quantifying changes, real-time learning or running queries and applications on evolving data. However, the samples need to be representative of the structural and topological properties, changing behaviors, distributions, and patterns in the networks. Here, we present some fast and effective memoryless sampling techniques biased to recency and the strength of changing relationships in an evolving network. They are also empirically proved to be closely preserving some important properties and distributions in various evolving networks. We also exploit them with the application perspective. Additionally, in this work, we introduce, analyze and recognize the significance of recurring links and develop a fast and scalable predictive model for recurring links in temporal network streams. Another contemporary application of network analytics is

Sleep is a composite of physiological and behavioral processes that undergo substantial changes during and after pregnancy. These changes might lead to sleep disorders and adverse pregnancy outcomes. Several studies have investigated this issue; however, they were restricted to subjective measurements or short-term actigraphy methods. This is insufficient for a longitudinal maternal sleep quality evaluation. A longitudinal study: 1) requires a long-term data collection approach to acquire data from everyday routines of mothers and 2) demands a sleep quality assessment method exploiting a large volume of multivariate data to assess sleep adaptations and overall sleep quality. In this paper, we present an Internet-of-Things-based long-term monitoring system to perform an objective sleep quality assessment. We conduct longitudinal monitoring, where 20 pregnant mothers are remotely monitored for six months of pregnancy and one month postpartum. To evaluate sleep quality adaptations, we: 1) extract several sleep attributes and study their variations during the monitoring and 2) propose a semi-supervised machine learning approach to create a personalized sleep model for each subject. The model provides an abnormality score, which allows an explicit representation of the sleep quality in a clinical routine, reflecting possible sleep quality degradation with respect to her own data. Sleep data of 13 participants (out of 20) are included in our analysis, as their data are adequate for the study, including 172.15±33.29 days of sleep data per person. Our fine-grained objective measurements indicate that the sleep duration and sleep efficiency are deteriorated in pregnancy and notably in postpartum. In comparison to the mid of the second trimester, the sleep model indicates the increase of sleep abnormality at the end of pregnancy (2.87 times) and postpartum (5.62 times). We also show that the model enables individualized and effective care for sleep disturbances during pregnancy, as compared to a baseline method.

Since the 1920’s, air traffic is becoming more prevalent by the year which results in a steady increase of the number of aircrafts roaming the airspace. This requires the expansion of the air surveillance systems in order to be able to manage each one of these aircrafts. Such an accommodation is planned to be implemented using different technologies and notably the Automatic Dependent Surveillance Broadcast (ADS-B) system. The ADS-B protocol is based on the idea that aircrafts as well as air traffic controllers communicate with each other using messages. However, for practicality reasons, those messages are not encrypted thus malicious messages can be injected. Hence, these attacks need to be detected to ensure the safety of the protocol. In this paper, we evaluate deep learning architectures for the purpose of detecting anomalous/malicious ADS-B messages, especially LSTM architectures which appear to be the most promising ones.