Responsible innovation, anticipation and responsiveness: case studies of algorithms in decision support in justice and security, and an exploration of potential, unintended, undesirable, higher-order effects

The collection and use of personal data on citizens in the design and deployment of algorithms in the domain of justice and security is a sensitive topic. Values like fairness, autonomy, privacy, accuracy, transparency and property are at stake. Negative examples of algorithms that propagate or exacerbate biases, inequalities or injustices have received ample attention, both in academia and in popular media. To supplement this view, we will discuss two positive examples of Responsible Innovation (RI): the design and deployment of algorithms in decision support, with good intentions and careful approaches. We then explore potential, unintended, undesirable, higher-order effects of algorithms—effects that may occur despite good intentions and careful approaches. We do that by engaging with anticipation and responsiveness, two key dimensions of Responsible Innovation. We close the paper with proposing a framework and a series of tentative recommendations to promote anticipation and responsiveness in the design and deployment of algorithms in decision support in the domain of justice and security.

[1]  Peter P.C.C. Verbeek,et al.  Beyond checklists: toward an ethical-constructive technology assessment , 2015 .

[2]  David J. Gunkel.  Mind the gap: responsible robotics and the problem of responsibility , 2020, Ethics and Information Technology.

[3]  Jeroen van den Hoven,et al.  Meaningful Human Control over Autonomous Systems: A Philosophical Account , 2018, Front. Robot. AI.

[4]  Predrag V. Klasnja,et al.  Value scenarios: a technique for envisioning systemic effects of new technologies , 2007, CHI Extended Abstracts.

[5]  Francesca Rossi,et al.  AI4People—An Ethical Framework for a Good AI Society: Opportunities, Risks, Principles, and Recommendations , 2018, Minds and Machines.

[6]  B. Mittelstadt.  Principles Alone Cannot Guarantee Ethical AI , 2019 .

[7]  J. Stilgoe,et al.  Developing a framework for responsible innovation* , 2013, The Ethics of Nanotechnology, Geoengineering and Clean Energy.

[8]  Jean Hartley,et al.  Case study research , 2004 .

[9]  Luciano Floridi,et al.  Translating Principles into Practices of Digital Ethics: Five Risks of Being Unethical , 2019, Philosophy & Technology.

[10]  Karthik Dinakar,et al.  Studying up: reorienting the study of algorithmic fairness around issues of power , 2020, FAT*.

[11]  Arie Rip,et al.  Constructive technology assessment and the methodology of insertion , 2019, Nanotechnology and Its Governance.

[12]  Matthias Scheutz,et al.  Against the moral Turing test: accountable design and the moral reasoning of autonomous systems , 2016, Ethics and Information Technology.

[13]  S. Brinkhoff.  Big Data Data Mining by the Dutch Police: Criteria for a Future Method of Investigation , 2017 .

[14]  A. Veenstra,et al.  The use of public sector data analytics in the Netherlands , 2020 .

[15]  Vincent Bonnemains,et al.  Embedded ethics: some technical and ethical challenges , 2018, Ethics and Information Technology.

[16]  James A. Reggia,et al.  Humanoid Cognitive Robots That Learn by Imitating: Implications for Consciousness Studies , 2018, Front. Robot. AI.

[17]  Diana Wright,et al.  Thinking in systems: a primer , 2012 .

[18]  I. Poel.  Society as a Laboratory to Experiment with New Technologies , 2017 .

[19]  Ben Shneiderman,et al.  Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy , 2020, Int. J. Hum. Comput. Interact..

[20]  Mariarosaria Taddeo,et al.  The ethics of algorithms: Mapping the debate , 2016, Big Data Soc..

[21]  Andrew Guthrie Ferguson,et al.  Policing Predictive Policing , 2016 .

[22]  Virginia Dignum,et al.  Responsible Artificial Intelligence: How to Develop and Use AI in a Responsible Way , 2019, Artificial Intelligence: Foundations, Theory, and Algorithms.

[23]  Ninja Marnau,et al.  Comments on the “Draft Ethics Guidelines for Trustworthy AI” by the High-LevelExpert Group on Artificial Intelligence. , 2019 .

[24]  Alan Borning,et al.  Value Sensitive Design and Information Systems , 2020, The Ethics of Information Technologies.

[25]  Paul Hayes,et al.  Algorithms and values in justice and security , 2020, AI & SOCIETY.

[26]  Arie Rip,et al.  Constructive Technology Assessment and Socio-Technical Scenarios. , 2008 .

[27]  B. Latour.  On Recalling Ant , 1999 .

[28]  Brent Mittelstadt,et al.  Principles alone cannot guarantee ethical AI , 2019, Nature Machine Intelligence.

[29]  P. Senge.  The fifth discipline : the art and practice of the learning organization/ Peter M. Senge , 1991 .

[30]  Armin Grunwald,et al.  Technology Assessment for Responsible Innovation , 2014 .

[31]  John Danaher,et al.  The Threat of Algocracy: Reality, Resistance and Accommodation , 2016, Philosophy & Technology.

[32]  Mark Coeckelbergh,et al.  Artificial Intelligence, Responsibility Attribution, and a Relational Justification of Explainability , 2019, Science and Engineering Ethics.

[33]  Reuben Binns,et al.  Fairness in Machine Learning: Lessons from Political Philosophy , 2017, FAT.

[34]  L. Floridi,et al.  The Ethics of Information , 2013, Dialogue.

[35]  Luciano Floridi,et al.  From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices , 2019, Science and Engineering Ethics.

[36]  Marijn Janssen,et al.  Digital platforms and responsible innovation: expanding value sensitive design to overcome ontological uncertainty , 2020, Ethics and Information Technology.

[37]  C. Argyris.  On organizational learning , 1993 .

[38]  S. Kulk,et al.  Juridische aspecten van algoritmen die besluiten nemen , 2020 .

[39]  Z. Kwee,et al.  Learning to do responsible innovation in industry: six lessons , 2020, Journal of Responsible Innovation.

[40]  Ibo van de Poel,et al.  An Ethical Framework for Evaluating Experimental Technology , 2015, Science and Engineering Ethics.

[41]  S. Kolassa.  Two Cheers for Rebooting AI: Building Artificial Intelligence We Can Trust , 2020 .

[42]  Aimee van Wynsberghe,et al.  Critiquing the Reasons for Making Artificial Moral Agents , 2018, Science and Engineering Ethics.

[43]  Thomas S. Woodson.  Weapons of math destruction , 2018, Journal of Responsible Innovation.

[44]  M. Steen,et al.  Advantages and disadvantages of societal engagement: a case study in a research and technology organization , 2020 .