Perceived benefits and barriers of a prototype early alert system to detect engagement and support 'at-risk' students: The teacher perspective

Abstract Given the focus on boosting retention rates and the potential benefits of proactive and early identification of students who may require support, higher education institutions are looking at the data already captured in university systems. Student early alert systems are part of formal, proactive, early intervention communication initiatives that institutions have put into place to help with the timely identification and intervention (alert) of at-risk students. The significance of student early alert systems is that support could be offered to high-risk students while they are still enrolled in the unit and able to influence their success/failure before the unit completes. Delivering timely interventions to students via a student early alert system typically requires teaching staff to identify at-risk students, and act upon that information in a way that would encourage students to change their behaviours. However, little is understood regarding teachers’ needs and attitudes towards the use of such a system. In the context of a prototype early alert system, this article reports the practices and opinions of a range of teaching staff across all faculties within an institution. The article sheds light on how teachers measure student performance and engagement in their units, and the perceived benefits and barriers of using the early alert system to identify and manage at-risk students.

[1]  Anastasios A. Economides,et al.  Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence , 2014, J. Educ. Technol. Soc..

[2]  Erik Duval,et al.  Learning Analytics Dashboard Applications , 2013 .

[3]  Adrian Kirkwood,et al.  Using technology for teaching and learning in higher education: a critical review of the role of evidence in informing practice , 2014 .

[4]  Danny Y. T. Liu,et al.  A learning analytics pilot in Moodle and its impact on developing organisational capacity in a university , 2017 .

[5]  V. Braun,et al.  Using thematic analysis in psychology , 2006 .

[6]  George Siemens,et al.  OnTask: Delivering Data-Informed, Personalized Learning Support Actions , 2018, J. Learn. Anal..

[7]  D. Weiss,et al.  Interrater reliability and agreement of subjective judgments , 1975 .

[8]  Billy Tak-Ming Wong,et al.  Learning analytics in higher education: an analysis of case studies , 2017 .

[9]  J. Wintrup.  Higher Education’s Panopticon? Learning Analytics, Ethics and Student Engagement , 2017 .

[10]  Wu He,et al.  Using Data Mining for Predicting Relationships between Online Question Theme and Final Grade , 2012, J. Educ. Technol. Soc..

[11]  D L Morgan,et al.  Practical Strategies for Combining Qualitative and Quantitative Methods: Applications to Health Research , 1998, Qualitative health research.

[12]  Geoffrey L. Cohen,et al.  Academic Tenacity: Mindsets and Skills that Promote Long-Term Learning. , 2014 .

[13]  Lorenzo Vigentini,et al.  Empowering Teachers to Personalize Learning Support , 2019, Utilizing Learning Analytics to Support Study Success.

[14]  Brian P. An.  The Role of Academic Motivation and Engagement on the Relationship between Dual Enrollment and Academic Performance , 2014 .

[15]  Mollie Dollinger,et al.  Working Together in Learning Analytics Towards the Co-Creation of Value , 2019, J. Learn. Anal..

[16]  Shane Dawson,et al.  Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan , 2012, J. Educ. Technol. Soc..

[17]  Glen R. Waddell,et al.  Modeling Retention at a Large Public University: Can At-Risk Students Be Identified Early Enough to Treat? , 2010 .

[18]  Hanan Ayad,et al.  Student success system: risk analytics and data visualization using ensembles of predictive models , 2012, LAK.

[19]  Vernon C. Smith,et al.  Predictive Modeling to Forecast Student Outcomes and Drive Effective Interventions in Online Community College Courses , 2012 .

[20]  Manfred Max Bergman,et al.  Advances in mixed methods research : theories and applications , 2008 .

[21]  Paul Franklin,et al.  Engaging Students through Communication and Contact: Outreach Can Positively Impact Your Students and You! , 2011 .

[22]  John W. Creswell,et al.  Using Mixed-Methods Sequential Explanatory Design: From Theory to Practice , 2006 .

[23]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[24]  Janet E. Hurn,et al.  Using learning analytics to predict (and improve) student success: a faculty perspective , 2013 .

[25]  James E. Willis,et al.  Ethics, Big Data, and Analytics: A Model for Application , 2013 .

[26]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[27]  Richard E. Boyatzis,et al.  Transforming Qualitative Information: Thematic Analysis and Code Development , 1998 .

[28]  Olga Viberg,et al.  The current landscape of learning analytics in higher education , 2018, Comput. Hum. Behav..

[29]  Anouschka van Leeuwen,et al.  Teachers’ perceptions of the usability of learning analytics reports in a flipped university course: when and how does information become actionable knowledge? , 2018, Educational Technology Research and Development.

[30]  Deborah Richards,et al.  An enhanced learning analytics plugin for Moodle: student engagement and personalised intervention , 2015 .

[31]  R. Hauser,et al.  High School Dropout, Graduation, and Completion Rates: Better Data, Better Measures, Better Decisions. , 2011 .

[32]  Dorit Maor,et al.  Exploring Learning Analytics as Indicators of Study Behaviour , 2012 .

[33]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[34]  H. Russell Bernard,et al.  Social Research Methods: Qualitative and Quantitative Approaches , 2000 .

[35]  P. Prinsloo,et al.  Learning Analytics , 2013 .

[36]  Hanan Ayad,et al.  Improving student success using predictive models and data visualisations , 2012 .

[37]  Linda Corrin,et al.  Completing the loop : returning learning analytics to teachers , 2014 .

[38]  Jafar Habibi,et al.  Using Educational Data Mining Methods to Study the Impact of Virtual Classroom in E-Learning , 2010, EDM.

[39]  George Siemens,et al.  Let’s not forget: Learning analytics are about learning , 2015 .

[40]  John G. Oetzel,et al.  Exploring the dimensions of organizational assimilation: Creating and validating a measure , 2003 .

[41]  Dragan Gasevic,et al.  Ethics and Privacy as Enablers of Learning Analytics , 2016 .

[42]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[43]  Mike Sharkey,et al.  Course correction: using analytics to predict course success , 2012, LAK '12.

[44]  J. Maxwell.  Understanding and Validity in Qualitative Research , 1992 .

[45]  Zdenek Zdráhal,et al.  Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment , 2013, LAK '13.

[46]  George Siemens,et al.  Ethical and privacy principles for learning analytics , 2014, Br. J. Educ. Technol..

[47]  C. Bryson.  Engagement through partnership: students as partners in learning and teaching in higher education , 2016 .

[48]  John W. Creswell,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2010 .

[49]  Doug Clow.  An overview of learning analytics , 2013 .

[50]  N. King,et al.  Interviews in Qualitative Research , 2010 .

[51]  Vincent Tinto,et al.  Enhancing student success: Taking the classroom success seriously , 2012 .

[52]  R. Schwager,et al.  Determinants of Students’ Success at University , 2014 .

[53]  Roslyn Cameron,et al.  A sequential mixed model research design: Design, analytical and display issues , 2009 .

[54]  Miguel Nussbaum,et al.  Teacher adoption of technology , 2013, Comput. Hum. Behav..

[55]  Alan Rubel,et al.  Student privacy in learning analytics: An information ethics perspective , 2014, Inf. Soc..

[56]  M. Maher,et al.  Retention and Attrition of Students in Higher Education: Challenges in Modern Times to What Works , 2013 .

[57]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[58]  W. F. Punch,et al.  Predicting student performance: an application of data mining methods with an educational Web-based system , 2003, 33rd Annual Frontiers in Education, 2003. FIE 2003..

[59]  M. Patton,et al.  Qualitative evaluation and research methods , 1992 .

[60]  Deborah Richards,et al.  Knowledge Acquisition for Learning Analytics: Comparing Teacher-Derived, Algorithm-Derived, and Hybrid Models in the Moodle Engagement Analytics Plugin , 2016, PKAW.

[61]  Deborah Richards,et al.  Validating the Effectiveness of the Moodle Engagement Analytics Plugin to Predict Student Academic Performance , 2015, AMCIS.

[62]  David Heath,et al.  Higher education teachers' experiences with learning analytics in relation to student retention , 2015 .

[63]  Monica W. Tracey,et al.  Exploring the relationship of ethics and privacy in learning analytics and design: implications for the field of educational technology , 2016 .

[64]  George Siemens,et al.  Penetrating the fog: analytics in learning and education , 2014 .

[65]  Adam J. Bridgeman,et al.  Data-Driven Personalization of Student Learning Support in Higher Education , 2017 .

[66]  Eitel J. M. Lauría,et al.  Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative , 2014, J. Learn. Anal..

[67]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..