Overview of the Author Obfuscation Task at PAN 2018: A New Approach to Measuring Safety

In this paper, we evaluate seven author obfuscation approaches which are supposed to automatically mask an author’s writing style in a given text to render automatic author identification impossible. The approaches are evaluated with regard to their safety, soundness, and sensibleness in terms of beating 44 author identification approaches, retaining the original meaning of the obfuscated text, and producing inconspicuous, human-readable obfuscations, respectively. Regarding the measurement of safety in particular, we introduce a set of new performance measures which are designed to render the performance of obfuscation approaches comparable as the numbers of author identification approaches and evaluation datasets increases, incorporating their respective performance and quality. Based on the new measures, we establish a world ranking of obfuscators.