Using engine signature to detect metamorphic malware

This paper introduces the "engine signature" approach to assist in detecting metamorphic malware by tracking it to its engine. More specifically, it presents and evaluates a code scoring technique for collecting forensic evidence from x86 code segments in order to get some measure of how likely they are to have been generated by some known instruction-substituting metamorphic engine. A prototype simulator that mimics real instruction-substituting metamorphic engines was implemented and used to conduct several experiments that evaluate the goodness of the scoring technique for given engine parameters. The technique was also used to successfully help track variants of W32.Evol to their engine.