High frequency patterns play a key role in the generation of adversarial examples

Abstract Deep neural networks (DNNs) have made significant progress in many fields, but the black-box nature hinders their application in critical areas. The emergence of adversarial examples raises serious questions about the safety of neural networks particularly. The interpretability of neural networks has become a research hotspot in the field of deep learning. It is significant to study what features DNNs have learned and how these features evolve within DNNs. Taking adversarial examples as the research object, we find that feature activations of all categories have relatively stable expectations at each layer, and adversarial examples have undergone a feature trajectory from origins to targets, generating the feature activations similar to those of the target category. Using image frequency spectrum, we prove that adversarial examples are mainly caused by the formation of high-frequency patterns widely present in the target category images. The conclusions of this paper contribute to a deeper understanding of feature learning and the generation of adversarial examples.