Selective manipulation of disentangled representations for privacy-aware facial image processing

. Camera sensors are increasingly being combined with machine learning to perform various tasks such as intelligent surveillance. Due to its computational complexity, most of these machine learning al-gorithms are offloaded to the cloud for processing. However, users are increasingly concerned about privacy issues such as function creep and malicious usage by third-party cloud providers. To alleviate this, we propose an edge-based filtering stage that removes privacy-sensitive attributes before the sensor data are transmitted to the cloud. We use state-of-the-art image manipulation techniques that leverage disentangled representations to achieve privacy filtering. We define opt-in and opt-out filter operations and evaluate their effectiveness for filtering pri-vate attributes from face images. Additionally, we examine the effect of naturally occurring correlations and residual information on filtering. We find the results promising and believe this elicits further research on how image manipulation can be used for privacy preservation.

[1]  Tao Wang,et al.  Distract Your Attention: Multi-Head Cross Attention Network for Facial Expression Recognition , 2021, Biomimetics.

[2]  Daniel Cohen-Or,et al.  Pivotal Tuning for Latent-based Editing of Real Images , 2021, ACM Trans. Graph..

[3]  Jing Wang,et al.  Face Privacy Protection Based on Attribute Manipulation , 2021, ICIT.

[4]  Pieter Simoens,et al.  Privacy Aware Person Detection in Surveillance Data , 2021, ArXiv.

[5]  Li Song,et al.  Personalized and Invertible Face De-identification by Disentangled Identity Information Manipulation , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[6]  Shobha Venkataraman,et al.  CrypTen: Secure Multi-Party Computation Meets Machine Learning , 2021, NeurIPS.

[7]  Yedid Hoshen,et al.  An Image is Worth More Than a Thousand Words: Towards Disentanglement in the Wild , 2021, NeurIPS.

[8]  Jaakko Lehtinen,et al.  Alias-Free Generative Adversarial Networks , 2021, NeurIPS.

[9]  Baoyuan Wu,et al.  TediGAN: Text-Guided Diverse Face Image Generation and Manipulation , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Daniel Cohen-Or,et al.  ReStyle: A Residual-Based StyleGAN Encoder via Iterative Refinement , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Daniel Cohen-Or,et al.  StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Ilya Sutskever,et al.  Learning Transferable Visual Models From Natural Language Supervision , 2021, ICML.

[13]  Daniel Cohen-Or,et al.  Designing an encoder for StyleGAN image manipulation , 2021, ACM Trans. Graph..

[14]  Kimmo Kärkkäinen,et al.  FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age , 2019, ArXiv.

[15]  R. Raskar,et al.  Privacy in Deep Learning: A Survey , 2020, ArXiv.

[16]  Aaron Hertzmann,et al.  GANSpace: Discovering Interpretable GAN Controls , 2020, NeurIPS.

[17]  Arun Ross,et al.  PrivacyNet: Semi-Adversarial Networks for Multi-Attribute Face Privacy , 2020, IEEE Transactions on Image Processing.

[18]  Tero Karras,et al.  Analyzing and Improving the Image Quality of StyleGAN , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Hamed Haddadi,et al.  Privacy and Utility Preserving Sensor-Data Transformations , 2019, Pervasive Mob. Comput..

[20]  Bolei Zhou,et al.  Interpreting the Latent Space of GANs for Semantic Face Editing , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  D. Tullsen,et al.  Shredder: Learning Noise Distributions to Protect Inference Privacy , 2019, ASPLOS.

[22]  Hamed Haddadi,et al.  Deep Private-Feature Extraction , 2018, IEEE Transactions on Knowledge and Data Engineering.

[23]  Peter Wonka,et al.  Image2StyleGAN: How to Embed Images Into the StyleGAN Latent Space? , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[24]  Farinaz Koushanfar,et al.  XONN: XNOR-based Oblivious Deep Neural Network Inference , 2019, IACR Cryptol. ePrint Arch..

[25]  Timo Aila,et al.  A Style-Based Generator Architecture for Generative Adversarial Networks , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Andrea Cavallaro,et al.  Mobile Sensor Data Anonymization , 2019 .

[27]  Mohammad H. Mahoor,et al.  AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild , 2017, IEEE Transactions on Affective Computing.

[28]  Ashwin Machanavajjhala,et al.  Olympus: Sensor Privacy through Utility Aware Obfuscation , 2019, Proc. Priv. Enhancing Technol..

[29]  Tim Verbelen,et al.  Privacy Aware Offloading of Deep Neural Networks , 2018, ICML 2018.

[30]  Andrea Cavallaro,et al.  Protecting Sensory Data against Sensitive Inferences , 2018, P2DS@EuroSys.

[31]  Anantha Chandrakasan,et al.  Gazelle: A Low Latency Framework for Secure Neural Network Inference , 2018, IACR Cryptol. ePrint Arch..

[32]  S. R. Livingstone,et al.  The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English , 2018, PloS one.

[33]  Yao Lu,et al.  Oblivious Neural Network Predictions via MiniONN Transformations , 2017, IACR Cryptol. ePrint Arch..

[34]  Michael Naehrig,et al.  CryptoNets: applying neural networks to encrypted data with high throughput and accuracy , 2016, ICML 2016.

[35]  Moni Naor,et al.  Theory and Applications of Models of Computation , 2015, Lecture Notes in Computer Science.

[36]  Cynthia Dwork,et al.  Differential Privacy: A Survey of Results , 2008, TAMC.

[37]  H. Theil On the Estimation of Relationships Involving Qualitative Variables , 1970, American Journal of Sociology.

[38]  H. Cramér Mathematical Methods of Statistics (PMS-9), Volume 9 , 1946 .