Comparing the effectiveness of commercial obfuscators against MATE attacks

The ability to protect software from malicious reverse engineering remains a challenge faced by commercial software companies who invest a large amount of resources in the development of their software product. In order to protect their investment from potential attacks such as illegal copying, tampering, and malicious reverse engineering, most companies utilize some type of protection software, also known as obfuscators, to create variants of their products that are more resilient to adversarial analysis. In this paper, we report on the effectiveness of different commercial obfuscators against traditional man-at-the-end (MATE) attacks where an adversary can utilize tools such as debuggers, disassemblers, and de-compilers as a legitimate end-user of a binary executable. Our case study includes four benchmark programs that have associated adversarial goals categorized as either comprehension or change tasks. We use traditional static and dynamic analysis techniques to identify the adversarial workload and outcomes before and after each program is transformed by a set of three commercial obfuscators. Our results confirm what is typically assumed: an adversary with a reasonable background in the computing disciplines can both comprehend and make changes to any of our completely unprotected programs using standard tools. Additionally, given the same skill set and attack approach, protected programs can still be probed to leak certain information, but none could be successfully altered and saved to create a cracked version. As a contribution, our methodology is unique compared to prior studies on obfuscation effectiveness in that we categorize adversarial skill and delineate program goals into comprehension and change ability, while considering the load time and overhead of obfuscated variants.