“The algorithm will screw you”: Blame, social actors and the 2020 A Level results algorithm on Twitter

In August 2020, the UK government and regulation body Ofqual replaced school examinations with automatically computed A Level grades in England and Wales. This algorithm factored in school attainment in each subject over the previous three years. Government officials initially stated that the algorithm was used to combat grade inflation. After public outcry, teacher assessment grades used instead. Views concerning who was to blame for this scandal were expressed on the social media website Twitter. While previous work used NLP-based opinion mining computational linguistic tools to analyse this discourse, shortcomings included accuracy issues, difficulties in interpretation and limited conclusions on who authors blamed. Thus, we chose to complement this research by analysing 18,239 tweets relating to the A Level algorithm using Corpus Linguistics (CL) and Critical Discourse Analysis (CDA), underpinned by social actor representation. We examined how blame was attributed to different entities who were presented as social actors or having social agency. Through analysing transitivity in this discourse, we found the algorithm itself, the UK government and Ofqual were all implicated as potentially responsible as social actors through active agency, agency metaphor possession and instances of passive constructions. According to our results, students were found to have limited blame through the same analysis. We discuss how this builds upon existing research where the algorithm is implicated and how such a wide range of constructions obscure blame. Methodologically, we demonstrated that CL and CDA complement existing NLP-based computational linguistic tools in researching the 2020 A Level algorithm; however, there is further scope for how these approaches can be used in an iterative manner.

[1]  J. Fischer,et al.  Critical reflections on three popular computational linguistic approaches to examine Twitter discourses , 2023, PeerJ Comput. Sci..

[2]  Zhibin Peng Cognitive Linguistics and Religious Language: An Introduction , 2022, Southern African Linguistics and Applied Language Studies.

[3]  Chris Edwards,et al.  Let the algorithm decide? , 2021, Commun. ACM.

[4]  Charles M. Mueller,et al.  Cognitive Linguistics and Religious Language , 2021 .

[5]  A. Aribowo,et al.  Implementation Of Text Mining For Emotion Detection Using The Lexicon Method (Case Study: Tweets About Covid-19) , 2021, Telematika.

[6]  A. Kelly A tale of two algorithms: The appeal and repeal of calculated grades systems in England and Ireland in 2020 , 2021, British Educational Research Journal.

[7]  Wouter van Atteveldt,et al.  The Validity of Sentiment Analysis: Comparing Manual Annotation, Crowd-Coding, Dictionary Approaches, and Machine Learning Algorithms , 2021, Communication Methods and Measures.

[8]  Aatu Liimatta,et al.  Chapter 5. Using lengthwise scaling to compare feature frequencies across text lengths on Reddit , 2020 .

[9]  Anis Sulalah THE SEMANTIC PROSODY ANALYSIS OF ‘INCREASE’ IN COVID-19: A CORPUS-BASED STUDY , 2020, Lire Journal (Journal of Linguistics and Literature).

[10]  Patrick J. Wolfe,et al.  Blame the algorithm? , 2020 .

[11]  Helen Smith Algorithmic bias: should students pay the price? , 2020, AI & SOCIETY.

[12]  K. Bhopal,et al.  The impact of COVID-19 on A level students in England , 2020 .

[13]  M A Muslim,et al.  Twitter text mining for sentiment analysis on government’s response to forest fires with vader lexicon polarity detection and k-nearest neighbor algorithm , 2020, Journal of Physics: Conference Series.

[14]  M. McGlashan Collective identity and discourse practice in the followership of the Football Lads Alliance on Twitter , 2020 .

[15]  S. Fadanelli,et al.  The representation of social actors in the tweets of Jair Messias Bolsonaro , 2020, Antares: letras e humanidades.

[16]  Dr.K.Mohan Kumar,et al.  Flock The Similar Users Of Twitter By Using Latent Dirichlet Allocation , 2019 .

[17]  Subhasree Sengupta,et al.  What are Academic Subreddits Talking About?: A Comparative Analysis of r/academia and r/gradschool , 2019, CSCW Companion.

[18]  R. Stine Sentiment Analysis , 2019, Annual Review of Statistics and Its Application.

[19]  Ben Wagner,et al.  Liable, but Not in Control? Ensuring Meaningful Human Agency in Automated Decision-Making Systems , 2019, Policy & Internet.

[20]  Michael Kranert,et al.  Doing Politics: Discursivity, performativity and mediation in political discourse , 2018 .

[21]  Silke Adam,et al.  Applying LDA Topic Modeling in Communication Research: Toward a Valid and Reliable Methodology , 2018 .

[22]  Ramona Kreis,et al.  #refugeesnotwelcome: Anti-refugee discourse on Twitter , 2017 .

[23]  Casey Fiesler,et al.  Understanding Diverse Interpretations of Animated GIFs , 2017, CHI Extended Abstracts.

[24]  N. Diakopoulos Accountability in algorithmic decision making , 2016, ACM Queue.

[25]  Zhao Jianqiang,et al.  Pre-processing Boosting Twitter Sentiment Analysis? , 2015, 2015 IEEE International Conference on Smart City/SocialCom/SustainCom (SmartCity).

[26]  Emilio Ferrara,et al.  Style in the Age of Instagram: Predicting Success within the Fashion Industry using Social Media , 2015, CSCW.

[27]  Paul Baker,et al.  Picking the right cherries? A comparison of corpus-based and qualitative analyses of news articles about masculinity , 2015 .

[28]  Ali Shojaie,et al.  Using Twitter for Demographic and Social Science Research: Tools for Data Collection and Processing , 2014, Sociological methods & research.

[29]  Lay-Ki Soon,et al.  Natural Language Processing for Sentiment Analysis: An Exploratory Analysis on Tweets , 2014, 2014 4th International Conference on Artificial Intelligence with Applications in Engineering and Technology.

[30]  Hope A. Olson,et al.  Quantitative "Versus" Qualitative Research: The Wrong Question , 2013 .

[31]  Grant Blank,et al.  Blurring the Boundaries? New social media, new social research: Developing a network to explore the issues faced by researchers negotiating the new research landscape of online social media platforms , 2013 .

[32]  Nina Wacholder,et al.  Identifying Sarcasm in Twitter: A Closer Look , 2011, ACL.

[33]  E. Bonelli Theoretical overview of the evolution of corpus linguistics , 2010 .

[34]  Paul Baker Sociolinguistics and Corpus Linguistics , 2010 .

[35]  S. Reeves,et al.  Discourse Analysis , 2018, Understanding Communication Research Methods.

[36]  M. Morris,et al.  Metaphors and the market: Consequences and preconditions of agent and object metaphors in stock market commentary , 2007 .

[37]  A. Goatly Washing the Brain - Metaphor and Hidden Ideology , 2007 .

[38]  Gerlinde Mautner Mining large corpora for social information: The case of elderly , 2007, Language in Society.

[39]  Teun A. van Dijk,et al.  Discourse, Ideology and Context , 2003 .

[40]  Mary Bucholtz,et al.  Reflexivity and Critique in Discourse Analysis , 2001 .

[41]  W. Clark Agents and Structures: Two Views of Preferences, Two Views of Institutions , 1998 .

[42]  Graeme D. Kennedy,et al.  Book Reviews: An Introduction to Corpus Linguistics , 1999, CL.

[43]  Tony McEnery,et al.  Corpus Linguistics: Method, Theory and Practice , 1996 .

[44]  Norman Fairclough,et al.  Critical Discourse Analysis and the Marketization of Public Discourse: The Universities , 1993 .

[45]  J. Weibler,et al.  Discourse , 1984, Language in Society.

[46]  R. Srinath,et al.  COVID-19 Vaccine –Public Sentiment Analysis Using Python’s Textblob Approach , 2021, International Journal of Current Research and Review.

[47]  Rachel Allison,et al.  “Something Seriously Wrong With U.S. Soccer”: A Critical Discourse Analysis of Consumers’ Twitter Responses to U.S. Soccer’s Girls’ Apparel Promotion , 2021, Journal of Sport Management.

[48]  Taryn Bernard The Discursive Representation of Social Actors in the Corporate Social Responsibility (CSR) and Integrated Annual (IA) Reports of Two South African Mining Companies , 2018 .

[49]  A. A. Allagbé,et al.  Principles, Theories and Approaches to Critical Discourse Analysis , 2018 .

[50]  V. Uma,et al.  An Extensive study of Sentiment Analysis tools and Binary Classification of tweets using Rapid Miner , 2018 .

[51]  Rayya Sulaiman Aljarallah A Critical Discourse Analysis of Twitter Posts on The Perspectives of Women Driving in Saudi Arabia , 2017 .

[52]  Eliza Rose A Month of Climate Change in Australia: A Corpus-Driven Analysis of Media Discourse , 2017 .

[53]  Tebogo Mogashoa Understanding Critical Discourse Analysis in Qualitative Research , 2014 .

[54]  Merja Mahrt,et al.  Twitter and society : an introduction , 2014 .

[55]  Bing Liu,et al.  Sentiment Analysis and Subjectivity , 2010, Handbook of Natural Language Processing.

[56]  Angela Morgan,et al.  Discourse Analysis: An Overview for the Neophyte Researcher , 2010 .

[57]  Theo van Leeuwen,et al.  Discourse and practice : new tools for critical discourse analysis , 2008 .

[58]  T. V. Leeuwen Discourse and Practice , 2008 .

[59]  R. Wodak Pragmatics and Critical Discourse Analysis. A cross-disciplinary inquiry. , 2007 .

[60]  Paul Baker Using Corpora in Discourse Analysis , 2006 .

[61]  Yael Maschler,et al.  Discourse Markers: Language, Meaning, and Context , 2005 .

[62]  G. Gaskell,et al.  Qualitative Researching with Text, Image and Sound , 2000 .

[63]  Y. Engeström,et al.  Activity theory and individual and social transformation. , 1999 .

[64]  Jan Blommaert,et al.  What is political discourse analysis , 1997 .

[65]  A. Leslie A theory of agency. , 1995 .

[66]  R. Scheines,et al.  Organizational Behavior and Human Decision Processes , 1977 .

[67]  M. Weber,et al.  Max Weber: ECONOMIC AND SOCIAL HISTORY , 1978 .