A Survey on Advanced Persistent Threats: Techniques, Solutions, Challenges, and Research Opportunities

Threats that have been primarily targeting nation states and their associated entities have expanded the target zone to include the private and corporate sectors. This class of threats, well known as advanced persistent threats (APTs), are those that every nation and well-established organization fears and wants to protect itself against. While nation-sponsored APT attacks will always be marked by their sophistication, APT attacks that have become prominent in corporate sectors do not make it any less challenging for the organizations. The rate at which the attack tools and techniques are evolving is making any existing security measures inadequate. As defenders strive to secure every endpoint and every link within their networks, attackers are finding new ways to penetrate into their target systems. With each day bringing new forms of malware, having new signatures and behavior that is close to normal, a single threat detection system would not suffice. While it requires time and patience to perform APT, solutions that adapt to the changing behavior of APT attacker(s) are required. Several works have been published on detecting an APT attack at one or two of its stages, but very limited research exists in detecting APT as a whole from reconnaissance to cleanup, as such a solution demands complex correlation and fine-grained behavior analysis of users and systems within and across networks. Through this survey paper, we intend to bring all those methods and techniques that could be used to detect different stages of APT attacks, learning methods that need to be applied and where to make your threat detection framework smart and undecipherable for those adapting APT attackers. We also present different case studies of APT attacks, different monitoring methods, and mitigation methods to be employed for fine-grained control of security of a networked system. We conclude this paper with different challenges in defending against APT and opportunities for further research, ending with a note on what we learned during our writing of this paper.

[1]  Sailik Sengupta,et al.  Moving Target Defense for the Placement of Intrusion Detection Systems in the Cloud , 2018, GameSec.

[2]  Dijiang Huang,et al.  MTD Analysis and evaluation framework in Software Defined Network (MASON) , 2018, SDN-NFV@CODASPY.

[3]  Úlfar Erlingsson,et al.  The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets , 2018, ArXiv.

[4]  Ruzanna Chitchyan,et al.  Data exfiltration: A review of external attack vectors and countermeasures , 2018, J. Netw. Comput. Appl..

[5]  Paulo Shakarian,et al.  Proactive identification of exploits in the wild through vulnerability mentions online , 2017, 2017 International Conference on Cyber Conflict (CyCon U.S.).

[6]  Roberto Baldoni,et al.  Survey on the Usage of Machine Learning Techniques for Malware Analysis , 2017, ArXiv.

[7]  Yuan Yan Tang,et al.  Security Evaluation of the Cyber Networks Under Advanced Persistent Threats , 2017, IEEE Access.

[8]  Xiaoyong Yuan.  PhD Forum: Deep Learning-Based Real-Time Malware Detection with Multi-Stage Analysis , 2017, 2017 IEEE International Conference on Smart Computing (SMARTCOMP).

[9]  Guowu Yang,et al.  Identifying APT Malware Domain Based on Mobile DNS Logging , 2017 .

[10]  Jassim Happa,et al.  Detecting disguised processes using application-behavior profiling , 2017, 2017 IEEE International Symposium on Technologies for Homeland Security (HST).

[11]  Sayak Ray,et al.  Malware detection using machine learning based analysis of virtual memory access patterns , 2017, Design, Automation & Test in Europe Conference & Exhibition (DATE), 2017.

[12]  Dijiang Huang,et al.  Dynamic Game based Security framework in SDN-enabled Cloud Networking Environments , 2017, SDN-NFV@CODASPY.

[13]  Cheng Lei,et al.  Optimal Strategy Selection for Moving Target Defense Based on Markov Game , 2017, IEEE Access.

[14]  Fenlin Liu,et al.  An SDN-Based Fingerprint Hopping Method to Prevent Fingerprinting Attacks , 2017, Secur. Commun. Networks.

[15]  Dong Seong Kim,et al.  Software Defined Stochastic Model for Moving Target Defense , 2016, AECIA.

[16]  Jarke J. van Wijk,et al.  Understanding the context of network traffic alerts , 2016, 2016 IEEE Symposium on Visualization for Cyber Security (VizSec).

[17]  Michele Colajanni,et al.  Analysis of high volumes of network traffic for Advanced Persistent Threat detection , 2016, Comput. Networks.

[18]  Dijiang Huang,et al.  SDN based Scalable MTD solution in Cloud Network , 2016, MTD@CCS.

[19]  Ahmad Diab,et al.  Darknet and deepnet mining for proactive cybersecurity threat intelligence , 2016, 2016 IEEE Conference on Intelligence and Security Informatics (ISI).

[20]  Igor Korkin,et al.  Acceleration of Statistical Detection of Zero-day Malware in the Memory Dump Using CUDA-enabled GPU Hardware , 2016, ArXiv.

[21]  Xinxin Niu,et al.  Detection of command and control in advanced persistent threat based on independent access , 2016, 2016 IEEE International Conference on Communications (ICC).

[22]  William M. S. Stout,et al.  Gathering threat intelligence through computer network deception , 2016, 2016 IEEE Symposium on Technologies for Homeland Security (HST).

[23]  Michele Colajanni,et al.  Countering Advanced Persistent Threats through security intelligence and big data analytics , 2016, 2016 8th International Conference on Cyber Conflict (CyCon).

[24]  William H. Sanders,et al.  Intrusion detection in enterprise systems by combining and clustering diverse monitor data , 2016, HotSoS.

[25]  Christoph Meinel,et al.  Advanced persistent threats: Behind the scenes , 2016, 2016 Annual Conference on Information Science and Systems (CISS).

[26]  Witold Kinsner,et al.  Detecting Advanced Persistent Threats using Fractal Dimension based Machine Learning Classification , 2016, IWSPA@CODASPY.

[27]  Katrin Franke,et al.  Malware Beaconing Detection by Mining Large-scale DNS Logs for Targeted Attack Identification , 2016 .

[28]  Jin B. Hong,et al.  Assessing the Effectiveness of Moving Target Defenses Using Security Models , 2016, IEEE Transactions on Dependable and Secure Computing.

[29]  Prasad Calyam,et al.  Frequency-minimal moving target defense using software-defined networking , 2016, 2016 International Conference on Computing, Networking and Communications (ICNC).

[30]  Ariana L. Johnson.  Cybersecurity for Financial Institutions: The Integral Role of Information Sharing in Cyber Attack Mitigation , 2016 .

[31]  Li Zhang,et al.  Detection of phishing emails using data mining algorithms , 2015, 2015 9th International Conference on Software, Knowledge, Information Management and Applications (SKIMA).

[32]  Radha Poovendran,et al.  A Game-Theoretic Approach to IP Address Randomization in Decoy-Based Cyber Defense , 2015, GameSec.

[33]  Jarke J. van Wijk,et al.  SNAPS: Semantic network traffic analysis through projection and selection , 2015, 2015 IEEE Symposium on Visualization for Cyber Security (VizSec).

[34]  Michael B. Crouse,et al.  Probabilistic Performance Analysis of Moving Target and Deception Reconnaissance Defenses , 2015, MTD@CCS.

[35]  Chih-Hung Hsieh,et al.  AD2: Anomaly detection on active directory log data for insider threat monitoring , 2015, 2015 International Carnahan Conference on Security Technology (ICCST).

[36]  Prasant Mohapatra,et al.  Dynamic defense strategy against advanced persistent threat with insiders , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[37]  B. Wu,et al.  Detecting APT Malware Infections Based on Malicious DNS and Traffic Analysis , 2015, IEEE Access.

[38]  Hsinchun Chen,et al.  Exploring threats and vulnerabilities in hacker web: Forums, IRC and carding shops , 2015, 2015 IEEE International Conference on Intelligence and Security Informatics (ISI).

[39]  Damir Delija,et al.  Advanced Persistent Threats - detection and defense , 2015, 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[40]  Florian Skopik,et al.  Combating advanced persistent threats: From network event correlation to incident detection , 2015, Comput. Secur..

[41]  Jonghyun Kim,et al.  Behavior-based anomaly detection on big data , 2015 .

[42]  Andrew Vance.  Flow based analysis of Advanced Persistent Threats detecting targeted attacks in cloud computing , 2014, 2014 First International Scientific-Practical Conference Problems of Infocommunications Science and Technology.

[43]  Ping Chen,et al.  A Study on Advanced Persistent Threats , 2014, Communications and Multimedia Security.

[44]  Nathaniel Evans,et al.  Multiple OS rotational environment an implemented Moving Target Defense , 2014, 2014 7th International Symposium on Resilient Control Systems (ISRCS).

[45]  Florian Skopik,et al.  Semi-synthetic data set generation for security software evaluation , 2014, 2014 Twelfth Annual International Conference on Privacy, Security and Trust.

[46]  Richard Kissel,et al.  Glossary of Key Information Security Terms , 2014 .

[47]  Harry G. Perros,et al.  SDN-based solutions for Moving Target Defense network protection , 2014, Proceeding of IEEE International Symposium on a World of Wireless, Mobile and Multimedia Networks 2014.

[48]  Ponnurangam Kumaraguru,et al.  Analyzing social and stylometric features to identify spear phishing emails , 2014, 2014 APWG Symposium on Electronic Crime Research (eCrime).

[49]  Edgar Toshiro Yano,et al.  Towards a Framework to Detect Multi-stage Advanced Persistent Threats Attacks , 2014, 2014 IEEE 8th International Symposium on Service Oriented System Engineering.

[50]  Babu M. Mehtre,et al.  Static Malware Analysis Using Machine Learning Methods , 2014, SNDS.

[51]  Tao Zhang,et al.  Bridging the Gap of Network Management and Anomaly Detection through Interactive Visualization , 2014, 2014 IEEE Pacific Visualization Symposium.

[52]  Jugal K. Kalita,et al.  Network Anomaly Detection: Methods, Systems and Tools , 2014, IEEE Communications Surveys & Tutorials.

[53]  Jian Pei,et al.  Email mining: tasks, common techniques, and tools , 2013, Knowledge and Information Systems.

[54]  William K. Robertson,et al.  Beehive: large-scale log analysis for detecting suspicious activity in enterprise networks , 2013, ACSAC.

[55]  Yasir Mehmood,et al.  Intrusion Detection System in Cloud Computing: Challenges and opportunities , 2013, 2013 2nd National Conference on Information Assurance (NCIA).

[56]  Dimitris Gritzalis,et al.  The Big Four - What We Did Wrong in Advanced Persistent Threat Detection? , 2013, 2013 International Conference on Availability, Reliability and Security.

[57]  Emilie Hogan,et al.  A graph analytic metric for mitigating advanced persistent threat , 2013, 2013 IEEE International Conference on Intelligence and Security Informatics.

[58]  Dipankar Dasgupta,et al.  Deriving behavior primitives from aggregate network features using support vector machines , 2013, 2013 5th International Conference on Cyber Conflict (CYCON 2013).

[59]  Richard J. Enbody,et al.  Targeted Cyberattacks: A Superset of Advanced Persistent Threats , 2013, IEEE Security & Privacy.

[60]  Xiaohua Yan,et al.  A Early Detection of Cyber Security Threats using Structured Behavior Modeling , 2013 .

[61]  Wei Wang,et al.  A Context-Based Detection Framework for Advanced Persistent Threats , 2012, 2012 International Conference on Cyber Security.

[62]  Ehab Al-Shaer,et al.  Openflow random host mutation: transparent moving target defense using software defined networking , 2012, HotSDN '12.

[63]  Michael E. Locasto,et al.  Software Diversity: Security, Entropy and Game Theory , 2012, HotSec.

[64]  Sushil Jajodia,et al.  Time-efficient and cost-effective network hardening using attack graphs , 2012, IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2012).

[65]  Vipin Kumar,et al.  Anomaly Detection for Discrete Sequences: A Survey , 2012, IEEE Transactions on Knowledge and Data Engineering.

[66]  Jason Hong,et al.  The state of phishing attacks , 2012, Commun. ACM.

[67]  Jiang Zhu,et al.  Helix: Unsupervised Grammar Induction for Structured Activity Recognition , 2011, 2011 IEEE 11th International Conference on Data Mining.

[68]  Atul Prakash,et al.  Distilling critical attack graph surface iteratively through minimum-cost SAT solving , 2011, ACSAC '11.

[69]  Stefan Savage,et al.  An analysis of underground forums , 2011, IMC '11.

[70]  Colin Tankard,et al.  Advanced Persistent threats and how to monitor and deter them , 2011, Netw. Secur..

[71]  Ralph Langner,et al.  Stuxnet: Dissecting a Cyberwarfare Weapon , 2011, IEEE Security & Privacy.

[72]  Salvatore J. Stolfo,et al.  Baiting Inside Attackers Using Decoy Documents , 2009, SecureComm.

[73]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[74]  Gabriel Maciá-Fernández,et al.  Anomaly-based network intrusion detection: Techniques, systems and challenges , 2009, Comput. Secur..

[75]  Heejo Lee,et al.  Scalable attack graph for risk assessment , 2009, 2009 International Conference on Information Networking.

[76]  Robert J. Ellison,et al.  Attack Trees , 2009, Encyclopedia of Biometrics.

[77]  Xinming Ou,et al.  Identifying Critical Attack Assets in Dependency Attack Graphs , 2008, ESORICS.

[78]  John Homer.  From Attack Graphs to Automated Configuration Management — An Iterative Approach , 2008 .

[79]  Heng Yin,et al.  Panorama: capturing system-wide information flow for malware detection and analysis , 2007, CCS '07.

[80]  Richard Lippmann,et al.  Practical Attack Graph Generation for Network Defense , 2006, 2006 22nd Annual Computer Security Applications Conference (ACSAC'06).

[81]  Karen Scarfone,et al.  Common Vulnerability Scoring System , 2006, IEEE Security & Privacy.

[82]  Xinming Ou,et al.  A scalable approach to attack graph generation , 2006, CCS '06.

[83]  Angelos D. Keromytis,et al.  Detecting Targeted Attacks Using Shadow Honeypots , 2005, USENIX Security Symposium.

[84]  Victoria J. Hodge,et al.  A Survey of Outlier Detection Methodologies , 2004, Artificial Intelligence Review.

[85]  Leslie Daigle.  WHOIS Protocol Specification , 2004, RFC.

[86]  Somesh Jha,et al.  Two formal analyses of attack graphs , 2002, Proceedings 15th IEEE Computer Security Foundations Workshop. CSFW-15.