Technology, Autonomy, and Manipulation

Since 2016, when the Facebook/Cambridge Analytica scandal began to emerge, public concern has grown around the threat of “online manipulation”. While these worries are familiar to privacy researchers, this paper aims to make them more salient to policymakers — first, by defining “online manipulation”, thus enabling identification of manipulative practices; and second, by drawing attention to the specific harms online manipulation threatens. We argue that online manipulation is the use of information technology to covertly influence another person’s decision-making, by targeting and exploiting their decision-making vulnerabilities. Engaging in such practices can harm individuals by diminishing their economic interests, but its deeper, more insidious harm is its challenge to individual autonomy. We explore this autonomy harm, emphasising its implications for both individuals and society, and we briefly outline some strategies for combating online manipulation and strengthening autonomy in an increasingly digital world.

[1]  J. Puusalu.  Antisocial media: How Facebook disconnects us and undermines democracy , 2020, European Journal of Cultural and Political Sociology.

[2]  Daniel Susser.  Notice After Notice-and-Consent: Why Privacy Disclosures Are Valuable Even If Consent Frameworks Aren't , 2019, Journal of Information Policy.

[3]  M. Elteren.  The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of PowerShoshanaZuboff. Public Affairs, 2019. , 2019, The Journal of American Culture.

[4]  Daniel Susser,et al.  Invisible Influence: Artificial Intelligence and the Ethics of Adaptive Choice Architectures , 2019, AIES.

[5]  Tal Z. Zarsky.  Privacy and Manipulation in the Digital Age , 2019, Theoretical Inquiries in Law.

[6]  Shoshana Zuboff.  The Age of Surveillance Capitalism: The Fight for a Human Future at the New Frontier of Power , 2019 .

[7]  Yeahin Pyo.  Yochai Benkler, Robert Faris, and Hal Roberts, Network Propaganda: Manipulation, Disinformation, and Radicalization in American Politics , 2019 .

[8]  Karen Yeung,et al.  ‘Hypernudge’: Big Data as a mode of regulation by design , 2016, The Social Power of Algorithms.

[9]  H. Nissenbaum,et al.  Online Manipulation: Hidden Influences in a Digital World , 2018 .

[10]  M Marjolein Lanzing,et al.  “Strongly Recommended” Revisiting Decisional Privacy to Judge Hypernudging in Self-Tracking Technologies , 2018, Philosophy & Technology.

[11]  Colin M. Gray,et al.  The Dark (Patterns) Side of UX Design , 2018, CHI.

[12]  Lee McGuigan,et al.  An impulse to exploit: the behavioral turn in data-driven marketing , 2018 .

[13]  Solon Barocas,et al.  The Intuitive Appeal of Explainable Machines , 2018 .

[14]  Sanne Kruikemeier,et al.  Online Political Microtargeting: Promises and Threats for Democracy , 2018 .

[15]  R. Noggle.  Manipulation, salience, and nudges , 2017, Bioethics.

[16]  M. Kosinski,et al.  Psychological targeting as an effective approach to digital mass persuasion , 2017, Proceedings of the National Academy of Sciences.

[17]  Sanne Kruikemeier,et al.  Behavioral Advertising : A Literature Review and Research Agenda , 2017 .

[18]  Bart Engelen,et al.  Judging Nudging: Answering the Manipulation Objection , 2017 .

[19]  Edoardo Lozza,et al.  The Ethics of Influence. Government in the Age of Behavioral Science , 2017 .

[20]  Alex Rosenblat,et al.  Algorithmic Labor and Information Asymmetries: A Case Study of Uber’s Drivers , 2016 .

[21]  Gérard Reach,et al.  Patient education, nudge, and manipulation: defining the ethical conditions of the person-centered model of care , 2016, Patient preference and adherence.

[22]  Damian Trilling,et al.  Should We Worry About Filter Bubbles? , 2016 .

[23]  Shoshana Zuboff,et al.  Big other: surveillance capitalism and the prospects of an information civilization , 2015, J. Inf. Technol..

[24]  Chris Arney.  Nudge: Improving Decisions about Health, Wealth, and Happiness , 2015 .

[25]  Mark Piper,et al.  Autonomy, Oppression, and Gender , 2014 .

[26]  Zeynep Tufekci,et al.  Engineering the public: Big data, surveillance and computational politics , 2014, First Monday.

[27]  Jonathan L. Zittrain.  Engineering an Election , 2014 .

[28]  I. Stole.  Persistent Pursuit of Personal Information: A Historical Perspective on Digital Advertising Strategies , 2014 .

[29]  Allen W. Wood,et al.  Coercion, Manipulation, Exploitation , 2014 .

[30]  Chris Peters,et al.  The daily you: How the new advertising industry is defining your identity and your worth , 2012, New Media Soc..

[31]  Ryan Calo.  Digital Market Manipulation , 2013 .

[32]  Neil M. Richards.  The Dangers of Surveillance , 2013 .

[33]  Daniel J. Solove,et al.  Privacy Self-Management and the Consent Dilemma , 2012 .

[34]  H. Nissenbaum.  A Contextual Approach to Privacy Online , 2011, Daedalus.

[35]  Evan Selinger,et al.  Is There a Right Way to Nudge? The Practice and Ethics of Choice Architecture , 2011 .

[36]  Yoni Van Den Eede,et al.  In Between Us: On the Transparency and Opacity of Technological Mediation , 2011 .

[37]  Dean Eckles,et al.  Selecting Effective Means to Any End: Futures and Ethics of Persuasion Profiling , 2010, PERSUASIVE.

[38]  J. Christman,et al.  The Politics of Persons: Individual Autonomy and Socio-historical Selves , 2009 .

[39]  L. Bovens.  The Ethics of Nudge , 2009 .

[40]  Robert Rosenberger,et al.  The sudden experience of the computer , 2009, AI & SOCIETY.

[41]  Tomas A. Lipinski,et al.  The Digital Person: Technology and Privacy in the Information Age , 2008 .

[42]  Catriona Mackenzie,et al.  Autonomy and the challenges to liberalism , 2008 .

[43]  Tal Zarsky.  Online Privacy, Tailoring, and Persuasion , 2006 .

[44]  J. Rachlinski.  Cognitive Errors, Individual Differences, and Paternalism , 2006 .

[45]  P. Verbeek.  What Things Do: Philosophical Reflections on Technology, Agency, and Design , 2005 .

[46]  A. Honneth,et al.  Autonomy, Vulnerability, Recognition, and Justice , 2005 .

[47]  Jeremy A. Blumenthal.  Does Mood Influence Moral Judgment?: An Empirical Test with Legal and Policy Implications , 2004 .

[48]  Lisa H. Schwartzman,et al.  Relational Autonomy: Feminist Perspectives on Autonomy, Agency, and the Social Self , 2002 .

[49]  Keith E. Stanovich,et al.  Individual differences in rational thought. , 1998 .

[50]  Thomas Scanlon,et al.  The Significance of Choice , 1986 .

[51]  S. Fender.  The New York Review of Books , 1986 .

[52]  Don McCarthy.  Ethics of Manipulation , 1977, Ethics & Medics.

[53]  Joseph Gray Jackson,et al.  Privacy and Freedom , 1968 .

[54]  J. Cyriax,et al.  Manipulation , 2018, Encyclopedia of Evolutionary Psychological Science.