Persuasion Meets AI: Ethical Considerations for the Design of Social Engineering Countermeasures

Privacy in Social Network Sites (SNSs) like Facebook or Instagram is closely related to people's self-disclosure decisions and their ability to foresee the consequences of sharing personal information with large and diverse audiences. Nonetheless, online privacy decisions are often based on spurious risk judgements that make people liable to reveal sensitive data to untrusted recipients and become victims of social engineering attacks. Artificial Intelligence (AI) in combination with persuasive mechanisms like nudging is a promising approach for promoting preventative privacy behaviour among the users of SNSs. Nevertheless, combining behavioural interventions with high levels of personalization can be a potential threat to people's agency and autonomy even when applied to the design of social engineering countermeasures. This paper elaborates on the ethical challenges that nudging mechanisms can introduce to the development of AI-based countermeasures, particularly to those addressing unsafe self-disclosure practices in SNSs. Overall, it endorses the elaboration of personalized risk awareness solutions as i) an ethical approach to counteract social engineering, and ii) as an effective means for promoting reflective privacy decisions.

[1]  L. Fewtrell,et al.  Risk communication. , 2005, Clinical medicine.

[2]  Jan Noyes,et al.  How does our perception of risk influence decision-making? Implications for the design of risk information , 2007 .

[3]  D. Boyd.  Social Network Sites as Networked Publics: Affordances, Dynamics, and Implications , 2010 .

[4]  Julita Vassileva,et al.  Motivating participation in social computing applications: a user modeling perspective , 2012, User Modeling and User-Adapted Interaction.

[5]  Yang Wang,et al.  "I regretted the minute I pressed share": a qualitative study of regrets on Facebook , 2011, SOUPS.

[6]  Konstantin Beznosov,et al.  Does my password go up to eleven?: the impact of password meters on password selection , 2013, CHI.

[7]  Yang Wang,et al.  Privacy nudges for social media: an exploratory Facebook study , 2013, WWW.

[8]  Juho Hamari,et al.  Social Motivations To Use Gamification: An Empirical Study Of Gamifying Exercise , 2013, ECIS.

[9]  Sherali Zeadally,et al.  Online deception in social media , 2014, Commun. ACM.

[10]  Bart P. Knijnenburg.  Information Disclosure Profiles for Segmentation and Recommendation , 2014 .

[11]  Darren Scott Appling,et al.  Cues to Deception in Social Media Communications , 2014, 2014 47th Hawaii International Conference on System Sciences.

[12]  Yiannis Kompatsiaris,et al.  PScore: A Framework for Enhancing Privacy Awareness in Online Social Networks , 2015, 2015 10th International Conference on Availability, Reliability and Security.

[13]  Edgar R. Weippl,et al.  Advanced social engineering attacks , 2015, J. Inf. Secur. Appl..

[14]  David Sánchez,et al.  Privacy Risk Assessment of Textual Publications in Social Networks , 2015, ICAART.

[15]  Aad P. A. van Moorsel,et al.  Nudging towards security: developing an application for wireless network selection for android phones , 2015, BCS HCI.

[16]  G. Loewenstein,et al.  Privacy and human behavior in the age of information , 2015, Science.

[17]  George R. S. Weir,et al.  Vulnerability to social engineering in social networks: a proposed user-centric framework , 2016, 2016 IEEE International Conference on Cybercrime and Computer Forensic (ICCCF).

[18]  Jan vom Brocke,et al.  Digital Nudging , 2016, Business & Information Systems Engineering.

[19]  L. Cranor,et al.  Nudges for Privacy and Security , 2017, ACM Comput. Surv..

[20]  Sanchit Alekh,et al.  Human Aspects and Perception of Privacy in Relation to Personalization , 2018, ArXiv.

[21]  Arnout R. H. Fischer,et al.  Perception of Product Risks , 2017 .

[22]  Janice Penni,et al.  The future of online social networks (OSN): A measurement analysis using social media tools and application , 2017, Telematics Informatics.

[23]  Paul De Bra,et al.  Challenges in User Modeling and Personalization , 2017, IEEE Intelligent Systems.

[24]  David E. Millard,et al.  The Cognitive Heuristics Behind Disclosure Decisions , 2017, SocInfo.

[25]  Maritta Heisel,et al.  At Your Own Risk: Shaping Privacy Heuristics for Online Self-disclosure , 2018, 2018 16th Annual Conference on Privacy, Security and Trust (PST).

[26]  Sourya Joyee De,et al.  On Consent in Online Social Networks: Privacy Impacts and Research Directions (Short Paper) , 2018, CRiSIS.

[27]  Sourya Joyee De,et al.  Privacy Risk Analysis to Enable Informed Privacy Settings , 2018, 2018 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW).

[28]  Karen Renaud,et al.  Ethical guidelines for nudging in information security & privacy , 2018, Int. J. Hum. Comput. Stud..

[29]  Hyrum S. Anderson,et al.  The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation , 2018, ArXiv.

[30]  Pieter H. Hartel,et al.  On the anatomy of social engineering attacks—A literature‐based dissection of successful attacks , 2018 .

[31]  Philipp K. Masur,et al.  Privacy and Self-Disclosure in the Age of Information , 2018, Situational Privacy and Self-Disclosure.

[32]  Hicham Hage,et al.  The Scourge of Online Deception in Social Networks , 2018, 2018 International Conference on Computational Science and Computational Intelligence (CSCI).

[33]  Hicham Hage,et al.  Manipulation and Malicious Personalization: Exploring the Self-Disclosure Biases Exploited by Deceptive Attackers on Social Media , 2019, Front. Artif. Intell..

[34]  Johanna Schäwel,et al.  Mastering the challenge of balancing self-disclosure and privacy in social media. , 2020, Current opinion in psychology.

[35]  Luisa Mich,et al.  Looking Inside the Black Box: Core Semantics Towards Accountability of Artificial Intelligence , 2019, From Software Engineering to Formal Methods and Tools, and Back.

[36]  Alessandro Acquisti,et al.  Can Privacy Nudges be Tailored to Individuals' Decision Making and Personality Traits? , 2019, WPES@CCS.

[37]  Ritesh Chugh,et al.  Do We Trust the Internet?: Ignorance and Overconfidence in Downloading and Installing Potentially Spyware-Infected Software , 2019, J. Glob. Inf. Manag..

[38]  Daniel Susser,et al.  Invisible Influence: Artificial Intelligence and the Ethics of Adaptive Choice Architectures , 2019, AIES.

[39]  Vincent C. Müller,et al.  Ethics of artificial intelligence and robotics , 2020 .

[40]  Stefan Stieglitz,et al.  Preventative Nudges: Introducing Risk Cues for Supporting Online Self-Disclosure Decisions , 2020, Inf..

[41]  Hicham Hage,et al.  Understanding the Landscape of Online Deception , 2021, Research Anthology on Fake News, Political Warfare, and Combatting the Spread of Misinformation.

[42]  Torben Jan Barev,et al.  Towards an Integrative Understanding of Privacy Nudging – Systematic Review and Research Agenda , 2020 .