Protection against indirect overflow attacks on pointers

Buffer overflow has accounted for a large fraction of Internet based attacks since 1988. Many solutions have been proposed to protect against a direct stack smashing attack overwriting a return address. Here, we target indirect buffer overflow attacks that overflow a buffer in memory to repoint a function pointer to the attacker's program. This type of attack could bypass most of the current stack protection mechanisms. Our proposed approach encrypts a function pointer before it is put into the memory and decrypts it before it is taken from the memory. Each function pointer is encrypted with a unique key that is randomized by the loader/linker for each program run. This leads to two desirable properties: (1) orthogonality of key space, (2) zero incremental knowledge gain for the adversary between two attacks on two different program runs. The key space orthogonality does not allow a one key compromise to propagate to other function pointers. The "zero knowledge gain" forces the adversary to compromise all (or most of) the keys in the same run. This is difficult since loader/linker based key randomization leads to a 232 iteration brute force attack on each key for a 32-bit architecture. This scheme was incorporated into GCC-3.0 on RedHat 7.0 Linux distribution. The performance overhead of this scheme is below 4.5% on Apache Web server version 1.3.22 with WebStone 2.5 as benchmark.