Detecting Log Anomalies with Multi-Head Attention (LAMA)

Anomaly detection is a crucial and challenging subject that has been studied within diverse research areas. In this work, we explore the task of log anomaly detection (especially computer system logs and user behavior logs) by analyzing logs’ sequential information. We propose LAMA, a multi-head attention based sequential model to process log streams as template activity (event) sequences. A next event prediction task is applied to train the model for anomaly detection. Extensive empirical studies demonstrate that our new model outperforms existing log anomaly detection methods including statistical and deep learning methodologies, which validate the effectiveness of our proposed method in learning sequence patterns of log data.

[1]  Salvatore J. Stolfo,et al.  Spectrogram: A Mixture-of-Markov-Chains Model for Anomaly Detection in Web Traffic , 2009, NDSS.

[2]  Yu Zhang,et al.  Log Clustering Based Problem Identification for Online Service Systems , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[3]  Zhi-Hua Zhou,et al.  Isolation Forest , 2008, 2008 Eighth IEEE International Conference on Data Mining.

[4]  Shilin He,et al.  Experience Report: System Log Analysis for Anomaly Detection , 2016, 2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE).

[5]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[6]  Feifei Li,et al.  DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning , 2017, CCS.

[7]  Ke Zhang,et al.  2016 Ieee International Conference on Big Data (big Data) Automated It System Failure Prediction: a Deep Learning Approach , 2022 .

[8]  Ling Huang,et al.  Large-Scale System Problems Detection by Mining Console Logs , 2009 .

[9]  Achmad Imam Kistijantoro,et al.  Log Anomaly Detection Using Adaptive Universal Transformer , 2019, 2019 International Conference of Advanced Informatics: Concepts, Theory and Applications (ICAICTA).

[10]  Andy Brown,et al.  Recurrent Neural Network Attention Mechanisms for Interpretable System Log Anomaly Detection , 2018, Proceedings of the First Workshop on Machine Learning for Computing Systems.

[11]  Qiang Fu,et al.  Mining Invariants from Console Logs for System Problem Detection , 2010, USENIX Annual Technical Conference.

[12]  Feifei Li,et al.  Spell: Online Streaming Parsing of Large Unstructured System Logs , 2019, IEEE Transactions on Knowledge and Data Engineering.