Abstraction by Term Rewriting for Malware Behavior Analysis - Extended Version

We propose a formal approach for behavioral analysis of programs based on dynamic analysis. It works by abstracting execution traces with respect to given behavior patterns in order to produce a high level representation of a program behavior and then, by comparing this abstract form to signatures defining reference abstract malicious behaviors. Abstraction is performed by term rewriting using rules on terms with variables, which enables to handle the data used by behavior functionalities. This technique allows us to deal with interleaved behaviors. Successfully applied to malware detection, it allows us in particular to model and detect information leak.