Learning Analytics at Low Cost: At-risk Student Prediction with Clicker Data and Systematic Proactive Interventions

While learning analytics (LA) practices have been shown to be practical and effective, most of them require a huge amount of data and effort. This paper reports a case study which demonstrates the feasibility of practising LA at a low cost for instructors to identify at-risk students in an undergraduate business quantitative methods course. Instead of using tracking data from a learning management system as predictive variables, this study utilised clicker responses as formative assessments, together with student demographic data and summative assessments. This LA practice makes use of free cloud services, Google Forms and Google Sheets in particular for collecting and analysing clicker data. Despite a small dataset being used, the LA implementation was effective in identifying at-risk students at an early stage. A systematic proactive advising approach is proposed as an intervention strategy based on students’ at-risk probability estimated by a prediction model. The result shows that the intervention success rate increases correspondingly with the number of interventions and the intervention effects on peer groups are far more successful than on individual students. Overall, the students’ pass rate in the study was 7% higher than that for the whole course. Practical recommendations and concerns about using linear regression and logistic regression for classification are also discussed.

[1]  Steven Lonn,et al.  Investigating student motivation in the context of a learning analytics intervention during a summer bridge program , 2015, Comput. Hum. Behav..

[2]  Carlos Delgado Kloos,et al.  Precise Effectiveness Strategy for analyzing the effectiveness of students with educational resources and activities in MOOCs , 2015, Comput. Hum. Behav..

[3]  Zdenek Zdráhal,et al.  Developing predictive models for early detection of at-risk students on distance learning modules , 2014, LAK Workshops.

[4]  Greg L. Alston,et al.  The methodology for the early identification of students at risk for failure in a professional degree program , 2014 .

[5]  Stephanie J. Jones,et al.  Technology Review: Virtual Intrusive Advising-Supporting Community College Students through Web-Based Synchronous Technologies , 2014 .

[6]  David C. Schwebel,et al.  Efficacy of Intrusively Advising First-Year Students via Frequent Reminders for Advising Appointments , 2008 .

[7]  A. O'connell.  Logistic Regression Models for Ordinal Response Variables , 2005 .

[8]  Marie Bienkowski,et al.  Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief , 2012 .

[9]  Walter R. Earl.  INTRUSIVE ADVISING OF FRESHMEN IN ACADEMIC DIFFICULTY , 1988 .

[10]  Dirk T. Tempelaar,et al.  In search for the most informative data for feedback generation: Learning analytics in a data-rich context , 2015, Comput. Hum. Behav..

[11]  Linda Corrin,et al.  Completing the Loop- Returning Meaningful Learning Analytics Data to Teachers , 2016 .

[12]  Eitel J. M. Lauría,et al.  Early Alert of Academically At-Risk Students: An Open Source Analytics Initiative , 2014, J. Learn. Anal..

[13]  Bart Rienties,et al.  Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK , 2016 .

[14]  Martin Hlosta,et al.  OU Analyse: analysing at-risk students at The Open University , 2015 .

[15]  Anastasios A. Economides,et al.  Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence , 2014, J. Educ. Technol. Soc..

[16]  Raymond Lister,et al.  Predicting student success using fine grain clicker data , 2014, ICER '14.

[17]  Guttorm Sindre,et al.  Evaluating the effectiveness of learning interventions: an information systems case study , 2003, ECIS.

[18]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..

[19]  Bart Rienties,et al.  Implementing a Learning Analytics Intervention and Evaluation Framework: what works? , 2017 .

[20]  Sarah Beth Wilson,et al.  Small Groups, Significant Impact: A Review of Peer-Led Team Learning Research with Implications for STEM Education Researchers and Faculty , 2016 .

[21]  Robert J. Dufresne,et al.  Classtalk: A classroom communication system for active learning , 1996, J. Comput. High. Educ..

[22]  Jennifer Shewmaker,et al.  The Case for Small Data in Higher Education , 2016, Intelligent Environments.

[23]  Farshid Marbouti,et al.  Models for early prediction of at-risk students in a course using standards-based grading , 2016, Comput. Educ..

[24]  Lorena Blasco-Arcas,et al.  Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance , 2013, Comput. Educ..

[25]  Samuel P. M. Choi,et al.  Modelling the Process of Learning Analytics Using a Reinforcement Learning Framework , 2018 .

[26]  Alyssa Friend Wise,et al.  Designing pedagogical interventions to support student use of learning analytics , 2014, LAK.

[27]  Doug Clow,et al.  The learning analytics cycle: closing the loop effectively , 2012, LAK.

[28]  Jill A. Marshall,et al.  Classroom Response Systems: A Review of the Literature , 2006 .

[29]  L. Pérez,et al.  Sorting, Supporting, Connecting, and Transforming: Intervention Strategies for Students at Risk , 1998 .

[30]  Eric Mazur,et al.  Peer Instruction: A User's Manual , 1996 .

[31]  Zdenek Zdráhal,et al.  Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment , 2013, LAK '13.

[32]  Matthew Fifolt,et al.  Academic Advising and First-Generation College Students: A Quantitative Study on Student Retention , 2013 .

[33]  Gwo-Jen Hwang,et al.  Objectives, methodologies and research issues of learning analytics , 2017, Interact. Learn. Environ..

[34]  Sridhar Iyer,et al.  iSAT: a visual learning analytics tool for instructors , 2016, Res. Pract. Technol. Enhanc. Learn..

[35]  L. Morris,et al.  Tracking student behavior, persistence, and achievement in online courses , 2005, Internet High. Educ..

[36]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[37]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[38]  D. Boud,et al.  Peer Learning in Higher Education: Learning from and with Each Other , 2014 .

[39]  V. Osborn.  Identifying at‐risk students in videoconferencing and web‐based distance education , 2001 .

[40]  Ottar Hellevik,et al.  Linear versus logistic regression when the dependent variable is a dichotomy , 2009 .

[41]  Nancy R. Glassman,et al.  Texting During Class: Audience Response Systems , 2015 .

[42]  Janie H. Wilson,et al.  Building rapport in the classroom and student outcomes. , 2012 .

[43]  J. Gillaspy,et al.  Brief Measure of Student-Instructor Rapport Predicts Student Success in Online Courses , 2013 .

[44]  Z. Higgs,et al.  Identifying students at risk for academic difficulty. , 1988, Journal of professional nursing : official journal of the American Association of Colleges of Nursing.

[45]  Robert J. Dufresne,et al.  Promoting active learning in large classes using a classroom communication system , 2008 .

[46]  Brian Dorn,et al.  Examining Instructor Use of Learning Analytics , 2017, CHI Extended Abstracts.

[47]  Chun-Yen Chang,et al.  Examining the Effects of Displaying Clicker Voting Results on High School Students’ Voting Behaviors, Discussion Processes, and Learning Outcomes , 2015 .