Designing for Contestation: Insights from Administrative Law

Algorithmic decision-making systems are increasingly being deployed to make, or to support humans to make, decisions that impact people’s lives in significant ways. Yet, decision subjects, those affected by algorithmic decisions, can be limited in their ability to contest these decisions. For example, the Education Value-Added Assessment System (EVAAS), a statistical method used to predict academic growth, was used by the Houston Independent School District to evaluate teachers’ performance and, in a number of cases, to terminate teachers’ contracts. Twelve teachers and the Houston Federation of Teachers successfully argued in court that the teachers’ constitutional right to due process was violated because they were unable to contest, or ‘meaningfully challenge’, the termination of their contracts because there was a ‘lack of sufficient information’ — the private company that designed EVAAS would not release the source codes or methodology used as they were proprietary trade secrets [1]. Even for decision subjects who are able to understand why a decision has been made and are provided with means to contest that decision, contestation systems can be seen as severely lacking [12]. Sarah Myers West studied content moderation across a number of social media platforms, most of which offered a way for users to contest a decision to remove their content from the platform [12]. Myers West reported user dissatisfaction with the contestation systems for a number of reasons, including a lack of clear instruction about how to lodge an appeal, no reply being received, no resolution being reached after a challenge has been lodged, and a lack of access to human intervention. These examples demonstrate that being able to challenge algorithmic decisions is important to decision subjects, yet numerous factors can limit a person’s ability to contest such decisions. We propose that administrative law systems, which were created to ensure that governments are kept accountable for their actions and decision making in relation to individuals [5, 6], can provide guidance on how to design contestation systems for algorithmic decision-making. There are similarities between government decision-making and algorithmic decision-making that suggest there is value in considering how the administrative law system enables contestation. For example, in both cases decision making can be said to occur ‘behind closed doors’, which limits