Metamorphic Malware Detection using Subgraph Matching

In the recent years, malicious codes called malware are having shown significant increase due to the code obfuscation to evade detection mechanisms. When the code obfuscation technique is applied to malwares, they can change their instruction sequence and also even their signature. These malwares which have same functionality and different appearance are able to evade signature-based AV products. Thus, AV venders paid large amount of cost to analyze and classify malware for generating the new signature. In this paper, we propose a novel approach for detecting metamorphic malwares. The proposed mechanism first converts malware's API call sequences to call graph through dynamic analysis. After that, the callgraph is converted to semantic signature using 128 abstract nodes. Finally, we extract all subgraphs and analyze how similar two malware's behaviors are through subgraph similarity. To validate proposed mechanism, we use 273 real-world malwares include obfuscated malware and analyze 10,100 comparison results. In the evaluation, all metamorphic malwares are classified correctly, and similar module behaviors among different malwares are also discovered.