Anomaly Detection and Sampling Cost Control via Hierarchical GANs

Anomaly detection incurs certain sampling and sensing costs and therefore it is of great importance to strike a balance between the detection accuracy and these costs. In this work, we study anomaly detection by considering the detection of threshold crossings in a stochastic time series without the knowledge of its statistics. To reduce the sampling cost in this detection process, we propose the use of hierarchical generative adversarial networks (GANs) to perform nonuniform sampling. In order to improve the detection accuracy and reduce the delay in detection, we introduce a buffer zone in the operation of the proposed GAN-based detector. In the experiments, we analyze the performance of the proposed hierarchical GAN detector considering the metrics of detection delay, miss rates, average cost of error, and sampling ratio. We identify the tradeoffs in the performance as the buffer zone sizes and the number of GAN levels in the hierarchy vary. We also compare the performance with that of a sampling policy that approximately minimizes the sum of average costs of sampling and error given the parameters of the stochastic process. We demonstrate that the proposed GAN-based detector can have significant performance improvements in terms of detection delay and average cost of error with a larger buffer zone but at the cost of increased sampling rates.

[1]  Tailai Wen,et al.  Time Series Anomaly Detection Using Convolutional Neural Networks and Transfer Learning , 2019, ArXiv.

[2]  Miao Xie,et al.  Anomaly Detection in Wireless Sensor Networks , 2013 .

[3]  Bo Hu,et al.  Active Anomaly Detection with Switching Cost , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[4]  Urbashi Mitra,et al.  Policy Design for Active Sequential Hypothesis Testing using Deep Learning , 2018, 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[5]  Lei Shi,et al.  MAD-GAN: Multivariate Anomaly Detection for Time Series Data with Generative Adversarial Networks , 2019, ICANN.

[6]  Bo Zong,et al.  A Deep Neural Network for Unsupervised Anomaly Detection and Diagnosis in Multivariate Time Series Data , 2018, AAAI.

[7]  Yuting Chen,et al.  Time Series Anomaly Detection Based on GAN , 2019, 2019 Sixth International Conference on Social Networks Analysis, Management and Security (SNAMS).

[8]  Kobi Cohen,et al.  Sequential Anomaly Detection Under a Nonlinear System Cost , 2019, IEEE Transactions on Signal Processing.

[9]  Senem Velipasalar,et al.  Deep Actor-Critic Reinforcement Learning for Anomaly Detection , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[10]  Kim-Kwang Raymond Choo,et al.  Outlier Dirichlet Mixture Mechanism: Adversarial Statistical Learning for Anomaly Detection in the Fog , 2019, IEEE Transactions on Information Forensics and Security.

[11]  Kobi Cohen,et al.  Deep Reinforcement One-Shot Learning for Change Point Detection , 2018, 2018 56th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[12]  Vincent W. S. Chan,et al.  Significant Sampling for Shortest Path Routing: A Deep Reinforcement Learning Solution , 2019, 2019 IEEE Global Communications Conference (GLOBECOM).

[13]  Ali Alqazzaz,et al.  AD-IoT: Anomaly Detection of IoT Cyberattacks in Smart City Using Machine Learning , 2019, 2019 IEEE 9th Annual Computing and Communication Workshop and Conference (CCWC).

[14]  Anton Kanev,et al.  Anomaly detection in wireless sensor network of the “smart home” system , 2017, 2017 20th Conference of Open Innovations Association (FRUCT).

[15]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[16]  Vincent W. S. Chan,et al.  Cognitive Network Management and Control with Significantly Reduced State Sensing , 2018, 2018 IEEE Global Communications Conference (GLOBECOM).