Dual-Encoding of Return Addresses for Detection and Defense against Stack Attacks

Program counter encoding is a programs' self-protection method by encrypting code pointers that comprise most control data. This paper presents return address dual-encoding improving defense and detection capabilities of program counter encoding for return addresses. Return address dual-encoding stores two versions of a return address encrypted by using two independent keys. It has the same effect using 64-bit keys suppressing the probability of successful attacks to . Two version of return address can be used for accurate detection of attacks by decrypting the encrypted addresses and comparing one with the other. We have implemented the idea in the GeC compiler for x86 and experimented defense capabilities and performance using the compiler. Compared with ProPolice embedded in GeC, ours could detect defend against two additional attack patterns. Performance overhead was less than 9%. Dual-encoding can be applied to all code pointers and some data pointers with reasonable performance overhead.