Automatic Detection and Analysis of Encrypted Messages in Malware

Encryption is increasingly used in network communications, especially by malicious software (malware) to hide its malicious activities and protect itself from being detected or analyzed. Understanding malware’s encryption schemes helps researchers better analyze its network protocol, and then derive the internal structure of the malware. However, current techniques of encrypted protocol analysis have a lot of limitations. For example, they usually require the encryption part being separated from message processing which is hardly satisfied in today’s malware, and they cannot provide detailed information about the encryption parameter such as the algorithm used and its secret key. Therefore, these techniques cannot fulfill the needs of today’s malware analysis.