A fast randomness test that preserves local detail

Randomness tests, sometimes called 'entropy' tests, can be a fast way to estimate whether a file is packed. However, most algorithms we have seen simply give an overall randomness value for the entire file. We present an algorithm that is not only fast, but preserves local detail, so a plot can be made of an entire file showing areas of high randomness and low randomness. Areas of low randomness (code and header, in the case of a PE file) show as lower points on the plot, whereas areas of high randomness (encrypted or compressed data) show as high areas. We present local detail-preserving entropy plots for a variety of packers, showing that many appear to pack files in a distinctive manner. This seems to be because compressed data and the code that unpacks it tends to be placed in the same relative location in the packed file, leading to a kind of signature based on a 'randomness signal'. We give algorithms which have shown early promise in comparing the entropy signals of various packed files. Finally, we are able to visualize the work of a packed program as it unpacks itself by placing breakpoints on decompression/ decryption loops, dumping memory and performing our detail-preserving randomness analysis on the dump.