CFA-Net: Controllable Face Anonymization Network with Identity Representation Manipulation

De-identification of face data has drawn increasing attention in recent years. It is important to protect people’s identities meanwhile keeping the utility of the data in many computer vision tasks. We propose a Controllable Face Anonymization Network (CFA-Net), a novel approach that can anonymize the identity of given faces in images and videos, based on a generator that can disentangle face identity from other image contents. We reach the goal of controllable face anonymization through manipulating identity vectors in the generator’s identity representation space. Various anonymized faces deriving from an original face can be generated through our method and maintain high similarity to the original image contents. Quantitative and qualitative results demonstrate our method’s superiority over literature models on visual quality and anonymization validity. Introduction Rapid developments of modern biometric technologies based on deep learning bring convenience to people, while abuse of them can cause serious legal and moral issues. It has become a significant concern for people to protect their biological information, especially facial information, from being misused by unauthorized software and malicious attackers. Face anonymization aims to hide the identity information of a given face to protect the privacy of the corresponding person. A suitable face anonymization method should meet the following requirements. • Anonymous effectiveness. A high de-identification success rate needs to be kept. An anonymized face should be able to prevent face recognition networks from matching it to the corresponding identity. • Data utility. Anonymized images should look realistic to human eyes and keep their utility for downstream tasks, such as detection, tracking, and action recognition. Meanwhile, the anonymized faces should maintain high similarity with the original images and retain their attributes as much as possible, such as pose, expression, etc. • Avoid identity leakage. The anonymized face has to avoid the risk of revealing the identity of other people. *These authors contributed equally. Figure 1: Our face anonymization results on CelebA-HQ dataset. The first column is the original images, and the remaining columns represent different anonymization effects. Identity-unrelated attributes such as pose, expression, hair cut are perfectly preserved, while identity features such as eyes, nose are modified. To be more concrete, the identity distances between the anonymized face and all existing faces in the database should stay far enough so that every identity in the database is secure. • Controllable. The identity of anonymized faces needs to be precisely controlled. In other words, modifications to the original face need to be controllable to meet different demands, which is an essential point that most literature approaches have not explored. The significance of controllable anonymization is as follows. 1. The identity distance between the anonymized face and the original face needs to be controllable so that the face recognition system can be bypassed in different threshold cases. 2. For aesthetic reasons, users want to pick their favorite face anonymization appearance. In order to meet the above requirements, the face anonymization method needs to control the extent and variety of anonymization flexibly and accurately. Traditional anonymization methods, including blurring, masking, pixelization, etc., can successfully hide the given ar X iv :2 10 5. 11 13 7v 2 [ cs .C V ] 1 0 O ct 2 02 1 identity, but they will seriously degrade the quality of the images and make them useless for downstream tasks. Performing direct face swapping can anonymize the target identity without damaging the image heavily, while it may cause harm to the person who provides the source identity and lead to ethical and legal issues. In recent years, some face anonymization methods (Gafni, Wolf, and Taigman 2019; Maximov, Elezi, and Leal-Taixé 2020) based on generative models have gained promising results in anonymizing images and videos. However, these models have relatively weak control over anonymized faces. (Gafni, Wolf, and Taigman 2019) can only yield stereotyped output for a single identity. Once the model is trained, only a fixed anonymized face can be generated for each face, and the anonymous degree cannot be adjusted. (Maximov, Elezi, and Leal-Taixé 2020) controls the anonymous results to some extent. However, it needs additional facial landmarks and masked images as supervisory signals, and its anonymization process depends on some reference identities of other people. In addition, the visual quality of its generated images is not good enough for human eyes. Our method explicitly separates the model training process and face anonymization into two independent stages and does not require any other reference face identity when anonymizing. Our approach can obtain an identity representation space on which we directly manipulate the identity of a given face with the maximum guarantee that the other image contents remain unchanged. Figure 1 shows our results on a high-resolution dataset, Celeba-HQ (Karras et al. 2017). With slight modifications to the original identity vector in the decoupled identity representation space, the face identity of original image is changed while identity-uncorrelated image contents are effectively preserved. Our contributions can be summarized as follow: • We propose a novel controllable face anonymization method, which can conduct flexible and precise face anonymization by directly manipulating the identity of the original face. • Our approach can obtain a highly controllable identity representation space through an identity disentanglement model. With the least modification on the other image contents, various anonymized faces can be generated via manipulating original face identity vector in this space, resulting in great improvement in controllability. • A large number of qualitative and quantitative experiments have verified the superiority of our method against literature methods from many perspectives. Related Work Face Anonymization Face anonymization aims to protect the private information of a given face. Simple face anonymization operations such as masking, mosaicing, blurring, etc., can destroy the usability of the face image. Nowadays, face anonymization emphasizes more on concealing the identity with least modification on other face contents. Early approaches (Gross et al. 2008; Samarzija and Ribaric 2014; Newton, Sweeney, and Malin 2005) used operations such as image distortions, image fusion, or distant face selection to achieve anonymization. Later, (Jourabloo, Yin, and Liu 2015) used an attribute classification network to extract the individual attribute parameters of the face and continuously updated them by optimization so that the generated face is not recognized as the original face. (Meden et al. 2017) proposed a generative model that combines k faces with different identities from the gallery data to generate anonymized faces. (Ren, Lee, and Ryoo 2018) proposed a privacy-preserving action detection model that uses a face modifier to change the identity of the input face and allows the action detection to proceed normally. (Sun et al. 2018a,b) adopted a two-stage training approach, with the first stage performing face identity replacement based on 3D face parameters or landmark detection and the second stage performing face image inpainting. Both methods use two-stage training and additional face annotations, which makes them less applicable. (Hukkelås, Mester, and Lindseth 2019) proposed a generative architecture for generating anonymized faces from noise, but its generation results are not realistic enough. Recently, (Gafni, Wolf, and Taigman 2019) proposed a de-identification method that can be applied to videos via an adversarial auto-encoder and a multi-level perceptual loss, but its generated anonymized faces are fixed for a given face. (Maximov, Elezi, and Leal-Taixé 2020) conduct diversified anonymization with a conditional GAN. However, anonymous faces generated by this method are unnatural and dissimilar from the original person. Although this method can generate multiple anonymized faces, the flexibility and continuity of control are not satisfactory. Unlike the face anonymization methods mentioned above, this paper proposes a controllable face anonymization approach, which decouples a highly controllable identity representation space. We can control the identity vectors in this space to allow flexible face anonymization while keeping other image contents unchanged maximally.