Edinburgh Research Explorer Opening the black box

ABSTRACT Building on the growing literature in algorithmic accountability, this paper investigates the use of a process visualisation technique known as the Petri net to achieve the aims of Privacy by Design. The strength of the approach is that it can help to bridge the knowledge gap that often exists between those in the legal and technical domains. Intuitive visual representations of the status of a system and the flow of information within and between legal and system models mean developers can embody the aims of the legislation from the very beginning of the software design process, while lawyers can gain an understanding of the inner workings of the software without needing to understand code. The approach can also facilitate automated formal verification of the models’ interactions, paving the way for machine-assisted privacy by design and, potentially, more general ‘compliance by design’. Opening up the ‘black box’ in this way could be a step towards achieving better algorithmic accountability.

[1]  Carmela Troncoso,et al.  Engineering Privacy by Design , 2011 .

[2]  Dave Evans,et al.  How the Next Evolution of the Internet Is Changing Everything , 2011 .

[3]  Danielle Keats Citron Open Code Governance , 2008 .

[4]  Tadao Murata,et al.  Petri nets: Properties, analysis and applications , 1989, Proc. IEEE.

[5]  Lawrence Lessig,et al.  Code - version 2.0 , 2006 .

[6]  Holger Hoffmann,et al.  Towards the Use of Software Requirement Patterns for Legal Requirements , 2012 .

[7]  John Danaher,et al.  The Threat of Algocracy: Reality, Resistance and Accommodation , 2016, Philosophy & Technology.

[8]  Lorrie Faith Cranor,et al.  Engineering Privacy , 2009, IEEE Transactions on Software Engineering.

[9]  G. Kendall,et al.  Using Foucault's Methods , 1998 .

[10]  K. Bamberger,et al.  Technologies of Compliance: Risk and Regulation in a Digital Age , 2009 .

[11]  Julie E. Cohen The regulatory state in the information age , 2016, Between Truth and Power.

[12]  Wolfgang Reisig,et al.  A Primer in Petri Net Design , 1992, Springer Compass International.

[13]  Daniel Oberle,et al.  Engineering Compliant Services: Advising Developers by Automating Legal Reasoning , 2012 .

[14]  Eoin Carolan,et al.  The continuing problems with online consent under the EU's emerging data protection principles , 2016, Comput. Law Secur. Rev..

[15]  Trevor J. M. Bench-Capon,et al.  Isomorphism and legal knowledge based systems , 1992, Artificial Intelligence and Law.

[16]  Alfons Bora,et al.  Final Report to the European Commission , 2004 .

[17]  James Grimmelmann,et al.  Regulation by Software , 2005 .

[18]  Dietmar P. F. Möller,et al.  Systems and Software Engineering , 2016 .

[19]  Jaap-Henk Hoepman,et al.  PDF hosted at the Radboud Repository of the Radboud University Nijmegen , 2022 .

[20]  Daniel Oberle,et al.  Volume 9, Issue 3, December, 2012 ENGINEERING COMPLIANT SOFTWARE: ADVISING DEVELOPERS BY AUTOMATING LEGAL REASONING , 2012 .

[21]  J. Reidenberg Lex Informatica: The Formulation of Information Policy Rules through Technology , 1997 .

[22]  Sol M. Shatz,et al.  A petri net framework for automated static analysis of Ada tasking behavior , 1988, J. Syst. Softw..

[23]  Annie I. Antón,et al.  Addressing Legal Requirements in Requirements Engineering , 2007, 15th IEEE International Requirements Engineering Conference (RE 2007).

[24]  Boualem Benatallah,et al.  A Petri Net-based Model for Web Service Composition , 2003, ADC.

[25]  Mike Wright,et al.  Petri net-based modelling of workflow systems: An overview , 2001, Eur. J. Oper. Res..

[26]  R. Kitchin,et al.  Thinking critically about and researching algorithms , 2014, The Social Power of Algorithms.

[27]  Ryan Calo,et al.  Code, Nudge, or Notice? , 2013 .

[28]  M. Hafiz A collection of privacy design patterns , 2006, PLoP '06.

[29]  Yao-Hua Tan,et al.  Modeling Deontic States in Petri Nets , 1995 .

[30]  Jörn Freiheit,et al.  Lexecute: visualisation and representation of legal procedures , 2006 .

[31]  C. Petri Kommunikation mit Automaten , 1962 .

[32]  M. Wendy Hennequin,et al.  The Future of the Internet and How to Stop It , 2011 .

[33]  Annie I. Antón,et al.  Towards Regulatory Compliance: Extracting Rights and Obligations to Align Requirements with Regulations , 2006, 14th IEEE International Requirements Engineering Conference (RE'06).

[34]  Eric Lachaud,et al.  Could the CE Marking Be Relevant to Enforce Privacy by Design in the Internet of Things , 2016 .

[35]  Bart Verheij,et al.  Argumentation Support Software: Boxes-and-Arrows and Beyond , 2007 .

[36]  Karsten Wolf,et al.  Transforming BPEL to Petri Nets , 2005, Business Process Management.

[37]  A. Cavoukian Privacy by Design: Origins, Meaning, and Prospects for Assuring Privacy and Trust in the Information Era , 2012 .

[38]  Ralph L. London,et al.  The current state of proving programs correct , 1972, ACM Annual Conference.

[39]  Paul De Hert,et al.  The new General Data Protection Regulation: Still a sound system for the protection of individuals? , 2016, Comput. Law Secur. Rev..

[40]  D. Citron Technological Due Process , 2007 .

[41]  Frank A. Pasquale The Black Box Society: The Secret Algorithms That Control Money and Information , 2015 .

[42]  Ronald Leenes,et al.  Privacy regulation cannot be hardcoded. A critical comment on the ‘privacy by design’ provision in data-protection law , 2014 .

[43]  Bill Lin,et al.  Software synthesis of process-based concurrent programs , 1998, Proceedings 1998 Design and Automation Conference. 35th DAC. (Cat. No.98CH36175).

[44]  Matthias Pocs Will the European Commission be able to standardise legal technology design without a legal method? , 2012, Comput. Law Secur. Rev..

[45]  Rinke Hoekstra,et al.  The LKIF Core Ontology of Basic Legal Concepts , 2007, LOAIT.

[46]  Burkhard Schafer,et al.  The Battle of the Precedents: Reforming Legal Education in Mexico Using Computer-Assisted Visualisation , 2013 .

[47]  Roger Brownsword So What Does the World Need Now? Reflections on Regulating Technologies , 2008 .

[48]  Joan Feigenbaum,et al.  Privacy Engineering for Digital Rights Management Systems , 2001, Digital Rights Management Workshop.