Learning analytics: challenges and limitations

ABSTRACT Learning analytic implementations are increasingly being included in learning management systems in higher education. We lay out some concerns with the way learning analytics – both data and algorithms – are often presented within an unproblematized Big Data discourse. We describe some potential problems with the often implicit assumptions about learning and learners – and indeed the tendency not to theorize learning explicitly – that underpin such implementations. Finally, we describe an attempt to devise our own analytics, grounded in a sociomaterial conception of learning. We use the data obtained to suggest that the relationships between learning and the digital traces left by participants in online learning are far from trivial, and that any analytics that relies on these as proxies for learning tends towards a behaviorist evaluation of learning processes.

[1]  Leslie Neal-Boylan,et al.  Overcoming Barriers. , 2015, Journal for nurses in professional development.

[2]  Alyssa Friend Wise,et al.  Learning analytics for online discussions: a pedagogical model for intervention with embedded and extracted analytics , 2013, LAK '13.

[3]  Ben Williamson,et al.  Digital education governance: An introduction , 2016 .

[4]  D. Garrison,et al.  Critical thinking, cognitive presence, and computer conferencing in distance education , 2001 .

[5]  Terrie Lynn Thompson,et al.  Criticality and the exercise of politeness in online spaces for professional learning , 2016, Internet High. Educ..

[6]  Francisco J. García-Peñalvo,et al.  Using Learning Analytics to improve teamwork assessment , 2015, Comput. Hum. Behav..

[7]  Doug Clow An overview of learning analytics , 2013 .

[8]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[9]  Lesley Gourlay,et al.  Student Engagement, ‘Learnification’ and the Sociomaterial: Critical Perspectives on Higher Education Policy , 2017 .

[10]  Katrien Verbert,et al.  Recommender Systems for Technology Enhanced Learning , 2014, Springer New York.

[11]  P. Goodyear,et al.  Beyond E-learning: a future for networked learning , 2001 .

[12]  Neil F. W. Saunders,et al.  Beyond marks: new tools to visualise student engagement via social networks , 2012 .

[13]  George Siemens,et al.  Guest Editorial - Learning and Knowledge Analytics , 2012, J. Educ. Technol. Soc..

[14]  Ángel Francisco Agudo-Peregrina,et al.  Assessing the suitability of student interactions from Moodle data logs as predictors of cross-curricular competencies , 2015, Comput. Hum. Behav..

[15]  Zdenek Zdrahal,et al.  Improving retention by identifying and supporting "at-risk" students , 2012 .

[16]  Rebecca Ferguson,et al.  Visualizing social learning ties by type and topic: rationale and concept demonstrator , 2013, LAK '13.

[17]  Linda Corrin,et al.  Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement , 2015 .

[18]  Jan H. F. Meyer,et al.  Overcoming Barriers to Student Understanding: Threshold Concepts and Troublesome Knowledge , 2006 .

[19]  Dokyun Lee,et al.  Impact of Recommender Systems on Sales Volume and Diversity , 2014, ICIS.

[20]  Vic Lally,et al.  Investigating patterns of interaction in networked learning and computer-supported collaborative learning: A role for Social Network Analysis , 2007, Int. J. Comput. Support. Collab. Learn..

[21]  Ben Williamson,et al.  Digital education governance: data visualization, predictive analytics, and ‘real-time’ policy instruments , 2016 .

[22]  Martin Davies,et al.  The Palgrave Handbook of Critical Thinking in Higher Education , 2015 .

[23]  D. Bligh Learning to teach in higher education , 1993 .

[24]  M. Newman,et al.  Finding community structure in very large networks. , 2004, Physical review. E, Statistical, nonlinear, and soft matter physics.

[25]  Tobias Hecking,et al.  Analysis of Dynamic Resource Access Patterns in Online Courses , 2014, J. Learn. Anal..

[26]  Peter H. Sawchuk,et al.  Emerging Approaches to Educational Research: Tracing the Socio-Material , 2011 .

[27]  Tom A. B. Snijders,et al.  Social Network Analysis , 2011, International Encyclopedia of Statistical Science.

[28]  Janet E. Hurn,et al.  Using learning analytics to predict (and improve) student success: a faculty perspective , 2013 .

[29]  B. Sangeetha,et al.  Who Wants to Be a Chemist? Formative Assessment Made Fun and Engaging. , 2018 .

[30]  L. Shulman Signature pedagogies in the professions , 2005, Daedalus.

[31]  P. Goodyear Psychological foundations for networked learning , 2001 .

[32]  Shane Dawson,et al.  Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan , 2012, J. Educ. Technol. Soc..

[33]  Larry Johnson,et al.  The NMC Horizon Report: 2012 Higher Education Edition. , 2012 .

[34]  Johann Ari Larusson,et al.  Monitoring student progress through their written "point of originality" , 2012, LAK.

[35]  Vernon C. Smith,et al.  Predictive Modeling to Forecast Student Outcomes and Drive Effective Interventions in Online Community College Courses , 2012 .

[36]  Terrie Lynn Thompson,et al.  Big data and learning analytics: Singular or plural? , 2017, First Monday.

[37]  George Siemens Connectivism: A Learning Theory for the Digital Age , 2004 .

[38]  Gabrielle Durepos Reassembling the Social: An Introduction to Actor‐Network‐Theory , 2008 .

[39]  Simon Buckingham Shum,et al.  Analytics for Learning and Becoming in Practice , 2014, ICLS.

[40]  Ram D. Gopal,et al.  Empirical Analysis of the Impact of Recommender Systems on Sales , 2010, J. Manag. Inf. Syst..

[41]  C. Lynch Big data: How do your data grow? , 2008, Nature.

[42]  S. Jaramillo,et al.  Adams Becker, S.; Cummins, M.; Davis, A.; Freeman, A.; Hall Giesinger, C.; Ananthanarayanan, V. NMC Horizon Report: 2017 Higher Education Edition. Austin, Texas: The New Media Consortium, 2017 , 2017 .

[43]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[44]  David Byrne,et al.  Complexity Theory and the Social Sciences : An Introduction , 2002 .

[45]  H. Rangwala,et al.  Learning Analytics in Higher Education , 2021, Advances in Electronic Government, Digital Divide, and Regional Development.

[46]  Rebecca Ferguson,et al.  Learning analytics to identify exploratory dialogue within synchronous text chat , 2011, LAK.

[47]  Ronnel B. King Learning and pedagogy : Asian perspectives , 2018 .

[48]  Trust A. Simpson Psychological Foundations of , 2007 .

[49]  Stephen Downes,et al.  INSTRUCTIONAL TECHNOLOGY AND DISTANCE LEARNING , 2005 .

[50]  Ben Williamson,et al.  Digital methodologies of education governance: Pearson plc and the remediation of methods , 2016 .

[51]  Bracha Shapira,et al.  Recommender Systems Handbook , 2015, Springer US.

[52]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..

[53]  David Heath,et al.  Let's talk learning analytics: A framework for implementation in relation to student retention , 2015 .

[54]  Marie Schmidt,et al.  Learning to Teach in Higher Education , 1992 .