Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Images

With an increasing number of users sharing information online, privacy implications entailing such actions are a major concern. For explicit content, such as user profile or GPS data, devices (e.g. mobile phones) as well as web services ( e.g. facebook) offer to set privacy settings in order to enforce the users’ privacy preferences. We propose the first approach that extends this concept to image content in the spirit of a Visual Privacy Advisor. First, we categorize personal information in images into 68 image attributes and collect a dataset, which allows us to train models that predict such information directly from images. Second, we run a user study to understand the privacy preferences of different users w.r.t. such attributes. Third, we propose models that predict user specific privacy score from images in order to enforce the users’ privacy preferences. Our model is trained to predict the user specific privacy risk and even outperforms the judgment of the users, who often fail to follow their own privacy preferences on image data.

[1]  Qi Tian,et al.  Principal Visual Word Discovery for Automatic License Plate Detection , 2012, IEEE Transactions on Image Processing.

[2]  Balachander Krishnamurthy,et al.  On the leakage of personally identifiable information via online social networks , 2009, CCRV.

[3]  Seong Joon Oh,et al.  I-Pic: A Platform for Privacy-Compliant Image Capture , 2016, MobiSys.

[4]  Paul A. Viola,et al.  Robust Real-Time Face Detection , 2001, International Journal of Computer Vision.

[5]  Carsten Eickhoff,et al.  Privacy Leakage through Innocent Content Sharing in Online Social Networks , 2016, ArXiv.

[6]  P. Rousseeuw Silhouettes: a graphical aid to the interpretation and validation of cluster analysis , 1987 .

[7]  Michael Jones,et al.  An improved deep learning architecture for person re-identification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Yuan Zhang,et al.  AppIntent: analyzing sensitive data transmission in android for privacy leakage detection , 2013, CCS.

[9]  Steven C. H. Hoi,et al.  Face Detection using Deep Learning: An Improved Faster RCNN Approach , 2017, Neurocomputing.

[10]  Anabel Quan-Haase,et al.  Information revelation and internet privacy concerns on social network sites: a case study of facebook , 2009, C&T.

[11]  Kristen LeFevre,et al.  Privacy wizards for social networking sites , 2010, WWW '10.

[12]  George Danezis Inferring privacy policies for social networking services , 2009, AISec '09.

[13]  Vitaly Shmatikov,et al.  De-anonymizing Social Networks , 2009, 2009 30th IEEE Symposium on Security and Privacy.

[14]  Ming Shao,et al.  What Do You Do? Occupation Recognition in a Photo via Social Context , 2013, 2013 IEEE International Conference on Computer Vision.

[15]  Qiang Wu,et al.  Learning-Based License Plate Detection Using Global and Local Features , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[16]  Alessandro Acquisti,et al.  Information revelation and privacy in online social networks , 2005, WPES '05.

[17]  Bernt Schiele,et al.  Loss Functions for Top-k Error: Analysis and Insights , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  David J. Crandall,et al.  Sensitive Lifelogs: A Privacy Analysis of Photos from Wearable Cameras , 2015, CHI.

[19]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Michael Roe,et al.  Scanning electronic documents for personally identifiable information , 2006, WPES '06.

[21]  Seyed-Mohsen Moosavi-Dezfooli,et al.  Universal Adversarial Perturbations , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Gang Wang,et al.  Seeing People in Social Context: Recognizing People and Social Relationships , 2010, ECCV.

[23]  Christoph Bier,et al.  Detection and Labeling of Personal Identifiable Information in E-mails , 2014, SEC.

[24]  David J. Crandall,et al.  Enhancing Lifelogging Privacy by Detecting Screens , 2016, CHI.

[25]  Herbert Burkert,et al.  Some Preliminary Comments on the DIRECTIVE 95/46/EC OF THE EUROPEAN PARLIAMENT AND OF THE COUNCIL of 24 October 1995 on the protection of individuals with regard to the processing of personal data and on the free movement of such data. , 1996 .

[26]  Vitaly Shmatikov,et al.  Can we still avoid automatic face detection? , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[27]  Adam N. Joinson,et al.  Development of measures of online privacy concern and protection for use on the Internet , 2007, J. Assoc. Inf. Sci. Technol..

[28]  Balachander Krishnamurthy,et al.  Characterizing privacy in online social networks , 2008, WOSN '08.

[29]  Tsuhan Chen,et al.  Clothing cosegmentation for recognizing people , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  Ning Zhang,et al.  Beyond frontal faces: Improving Person Recognition using multiple cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Koby Crammer,et al.  A Family of Additive Online Algorithms for Category Ranking , 2003, J. Mach. Learn. Res..

[32]  Yang Song,et al.  Tour the world: Building a web-scale landmark recognition engine , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Cornelia Caragea,et al.  Privacy Prediction of Images Shared on Social Media Sites Using Deep Features , 2015, ArXiv.

[34]  Ananthram Swami,et al.  Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks , 2015, 2016 IEEE Symposium on Security and Privacy (SP).

[35]  Sanjeev J. Koppal,et al.  Privacy preserving optics for miniature vision sensors , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Balachander Krishnamurthy,et al.  Privacy leakage vs . Protection measures : the growing disconnect , 2011 .

[37]  Jian Pei,et al.  Preserving Privacy in Social Networks Against Neighborhood Attacks , 2008, 2008 IEEE 24th International Conference on Data Engineering.

[38]  Xin Wang,et al.  Using Data Mining Methods to Predict Personally Identifiable Information in Emails , 2008, ADMA.

[39]  Christian Bauckhage,et al.  Age Recognition in the Wild , 2010, 2010 20th International Conference on Pattern Recognition.

[40]  Sei-Wang Chen,et al.  Automatic license plate recognition , 2004, IEEE Transactions on Intelligent Transportation Systems.

[41]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[42]  Krishna P. Gummadi,et al.  Analyzing facebook privacy settings: user expectations vs. reality , 2011, IMC '11.

[43]  Ashwin Machanavajjhala,et al.  MarkIt: privacy markers for protecting visual secrets , 2014, UbiComp Adjunct.

[44]  Starr Roxanne Hiltz,et al.  Trust and Privacy Concern Within Social Networking Sites: A Comparison of Facebook and MySpace , 2007, AMCIS.

[45]  Jesús Martínez del Rincón,et al.  Recurrent Convolutional Network for Video-Based Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Saikat Guha,et al.  NOYB: privacy in online social networks , 2008, WOSN '08.

[47]  Seong Joon Oh,et al.  Faceless Person Recognition: Privacy Implications in Social Media , 2016, ECCV.

[48]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[49]  Ming Li,et al.  FindU: Privacy-preserving personal profile matching in mobile social networks , 2011, 2011 Proceedings IEEE INFOCOM.

[50]  Carman Neustaedter,et al.  Blur filtration fails to preserve privacy for home-based video conferencing , 2006, TCHI.

[51]  A. Lenhart,et al.  Teens, privacy and online social networks: How teens manage their online identities and personal information in the age of MySpace , 2007 .