A Survey on malware analysis and mitigation techniques

Abstract In recent days, malwares are advanced, sophisticatedly engineered to attack the target. Most of such advanced malwares are highly persistent and capable of escaping from the security systems. This paper explores such an advanced malware type called Advanced Persistent Threats (APTs). APTs pave the way for most of the Cyber espionages and sabotages. APTs are highly sophisticated, target specific and operate in a stealthy mode till the target is compromised. The intention of the APTs is to deploy target specific automated malwares in a host or network to initiate an on-demand attack based on continuous monitoring. Encrypted covert communication and advanced, sophisticated attack techniques make the identification of APTs more challenging. Conventional security systems like antivirus, anti-malware systems which depend on signatures and static analysis fail to identify these APTs. The Advanced Evasive Techniques (AET) used in APTs are capable of bypassing the stateful firewalls housed in the enterprise choke points at ease. Hence, this paper presents a detailed study on sophisticated attack and evasion techniques used by the contemporary malwares. Furthermore, existing malware analysis techniques, application hardening techniques and CPU assisted application security schemes are also discussed. Finally, the study concludes by presenting the System and Network Security Design (SNSD) using existing mitigation techniques.