Analytics for Student Engagement

Volumes of detailed information are now unobtrusively collected as students use learning management systems and digital learning environments in their studies. This significantly elevates opportunities to better understand how students learn. The learning analytics community is exploring these data to describe learning processes [117] and ground recommendations for improved learning environments [8, 102, 139]. One challenge in this work is need for more and more detailed information about each student’s learning processes to mine for developing useful and timely feedback for students and teachers [150]. Student engagement is a strong focus in this work. Research consistently shows positive relationships between engagement and academic success (e.g., [68, 111]). However, to construct learning analytics describing student engagement and recommending more productive forms of engagement, the field needs to better understand what student engagement means, how it can be observed online and quantified, and how it relates to learning processes and achievement. We review literatures describing student engagement and its relations to learning focusing on engagement in online and distance learning in postsecondary education. We catalog conceptualizations, measurement approaches and benefits of student engagement for learning and academic achievement. Through lenses of learning analytics and learning science, we map the evolution of analytics about student engagement and propose future research to bridge the learning analytics—learning science gap. We note challenges of tracking and supporting student engagement and recommend strategies that are predicted to facilitate positive long-term change.

[1]  Gerard H. Seijts,et al.  Distinguished Scholar Invited Essay , 2016 .

[2]  G. Burkholder,et al.  Emerging Evidence Regarding the Roles of Emotional, Behavioural, and Cognitive Aspects of Student Engagement in the Online Classroom , 2017 .

[3]  Anouschka van Leeuwen,et al.  Supporting teachers in guiding collaborating students: Effects of learning analytics in CSCL , 2014, Comput. Educ..

[4]  Andrew D. Ho,et al.  Changing “Course” , 2014 .

[5]  Erik Duval,et al.  Learning dashboards: an overview and future research opportunities , 2013, Personal and Ubiquitous Computing.

[6]  F. Oort,et al.  The Influence of Affective Teacher–Student Relationships on Students’ School Engagement and Achievement , 2011 .

[7]  J. Finn,et al.  Student engagement: What is it? Why does it matter? , 2012 .

[8]  Peter Brusilovsky,et al.  QuizMap: Open Social Student Modeling and Adaptive Navigation Support with TreeMaps , 2011, EC-TEL.

[9]  Philip J. Guo,et al.  How video production affects student engagement: an empirical study of MOOC videos , 2014, L@S.

[10]  Elina E. Ketonen,et al.  Am I in the right place?: Academic engagement and study success during the first years at university , 2016 .

[11]  Daniel M. Russell,et al.  Student skill and goal achievement in the mapping with google MOOC , 2014, L@S.

[12]  Dwen-Ren Tsai,et al.  Social interaction feedback system for the smart classroom , 2012, 2012 IEEE International Conference on Consumer Electronics (ICCE).

[13]  Robert M. Gonyea,et al.  Unmasking the Effects of Student Engagement on First-Year College Grades and Persistence , 2008 .

[14]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[15]  K. J. Cairns,et al.  Negative Effects of Praise on Skilled Performance , 1990 .

[16]  Anne Trumbore,et al.  Rules of Engagement: Strategies to Increase Online Engagement at Scale , 2014 .

[17]  Kerri-Lee Krause,et al.  Students’ engagement in first‐year university , 2008 .

[18]  Rémi Venant,et al.  Evaluation of Learner Performance During Practical Activities: An Experimentation in Computer Education , 2016, 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT).

[19]  Robert A. Smallwood,et al.  Assessment Measures: CLASSE--The Class-Level Survey of Student Engagement. , 2005 .

[20]  Lauri Nummenmaa,et al.  Experienced emotions, emotion regulation and student activity in a web-based learning environment , 2004 .

[21]  Shanna Smith Jaggars,et al.  Creating an Effective Online Instructor Presence. , 2013 .

[22]  Ke Zhang,et al.  Revealing Online Learning Behaviors and Activity Patterns and Making Predictions with Data Mining Techniques in Online Teaching , 2008 .

[23]  W. Briggs,et al.  A Measure of College Student Course Engagement , 2005 .

[24]  Philip H. Winne,et al.  Learning Analytics for Self-Regulated Learning , 2017 .

[25]  N. Schwarz Emotion, cognition, and decision making , 2000 .

[26]  Susan Lowes,et al.  Exploring the Link between Online Behaviours and Course Performance in Asynchronous Online High School Courses , 2015, J. Learn. Anal..

[27]  Marek Hatala,et al.  The role of achievement goal orientations when studying effect of learning analytics visualizations , 2016, LAK.

[28]  Barbara A. Greene,et al.  Influences on achievement : Goals, perceived ability, and cognitive engagement , 1996 .

[29]  Alyssa Friend Wise,et al.  Designing pedagogical interventions to support student use of learning analytics , 2014, LAK.

[30]  Marek Hatala,et al.  Penetrating the black box of time-on-task estimation , 2015, LAK.

[31]  Fengfeng Ke,et al.  The effect of peer feedback for blogging on college students' reflective learning processes , 2008, Internet High. Educ..

[32]  David Zyngier,et al.  (Re)conceptualising Student Engagement: Doing Education Not Doing Time. , 2008 .

[33]  Maren Scheffel,et al.  Awareness Is Not Enough: Pitfalls of Learning Analytics Dashboards in the Educational Practice , 2017, EC-TEL.

[34]  J. Finn Withdrawing From School , 1989 .

[35]  Samuel N. Kamin,et al.  Developing Device-independent Applications for Active and Collaborative Learning with the SLICE Framework , 2012 .

[36]  Daniel E. Atkins,et al.  Higher Education in the Digital Age: Technology Issues and Strategies for American Colleges and Universities (ACE/Praeger Series on Higher Education) , 2002 .

[37]  Khe Foon Hew,et al.  Promoting engagement in online courses: What strategies can we learn from three highly rated MOOCS , 2016, Br. J. Educ. Technol..

[38]  Sidney K. D'Mello,et al.  Emotional Learning Analytics , 2017 .

[39]  Shui-fong Lam,et al.  Understanding student engagement with a contextual model , 2012 .

[40]  Teresa Petty,et al.  Investigating Student Engagement in an Online Mathematics Course through Windows into Teaching and Learning , 2013 .

[41]  D. Stipek,et al.  Good Instruction Is Motivating , 2002 .

[42]  V. Dennen,et al.  Instructor–Learner Interaction in Online Courses: The relative perceived importance of particular instructor actions on performance and satisfaction , 2007 .

[43]  Lenh Hung Son Do Supporting Reflection and Classroom Orchestration with Tangible Tabletops , 2012 .

[44]  E. A. Locke,et al.  The effects of simultaneous learning and performance goals on performance: An inductive exploration , 2015 .

[45]  Doug Clow,et al.  The learning analytics cycle: closing the loop effectively , 2012, LAK.

[46]  Sean B. Eom,et al.  The Determinants of Students' Perceived Learning Outcomes and Satisfaction in University Online Education: An Empirical Investigation* , 2006 .

[47]  Jinan Banna,et al.  Interaction matters: Strategies to promote engaged learning in an online introductory nutrition course , 2015, Journal of online learning and teaching.

[48]  John C. Whitmer,et al.  Logging On to Improve Achievement: Evaluating the Relationship between Use of the Learning Management System, Student Characteristics, and Academic Achievement in a Hybrid Large Enrollment Undergraduate Course , 2013 .

[49]  Rebecca Ferguson,et al.  Examining engagement: analysing learner subpopulations in massive open online courses (MOOCs) , 2015, LAK.

[50]  Abelardo Pardo,et al.  Exploring the relation between self-regulation, online activities, and academic performance: a case study , 2016, LAK.

[51]  D. Bolliger,et al.  Engagement Matters: Student Perceptions on the Importance of Engagement Strategies in the Online Learning Environment. , 2018 .

[52]  Lynn McAlpine,et al.  Reflection: issues related to improving professors' teaching and students' learning , 2000 .

[53]  Chia-Ching Lin,et al.  Participatory learning through behavioral and cognitive engagements in an online collective information searching activity , 2012, Int. J. Comput. Support. Collab. Learn..

[54]  Rebecca Ferguson,et al.  Social learning analytics: five approaches , 2012, LAK.

[55]  Charles W. Anderson,et al.  Task Engagement and Conceptual Change in Middle School Science Classrooms , 1993 .

[56]  Elizabeth Koh,et al.  Fostering 21st century literacies through a collaborative critical reading and learning analytics environment: user-perceived benefits and problematics , 2016, LAK.

[57]  Claus Zinn,et al.  How did the e-learning session go? The Student Inspector , 2007, AIED.

[58]  Yeonjeong Park,et al.  Educational Dashboards for Smart Learning: Review of Case Studies , 2014, ICSLE.

[59]  Ido Roll,et al.  Understanding, evaluating, and supporting self-regulated learning using learning analytics , 2015, J. Learn. Anal..

[60]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[61]  Reinhard Pekrun,et al.  Boredom in achievement settings: Exploring control–value antecedents and performance outcomes of a neglected emotion. , 2010 .

[62]  A. Furnham Response bias, social desirability and dissimulation , 1986 .

[63]  E. Duval Attention please!: learning analytics for visualization and recommendation , 2011, LAK.

[64]  Anne-Marie Nuttall,et al.  Student Engagement Patterns over the Duration of Level 1 and Level 3 Geography Modules: Influences on Student Attendance, Performance and Use of Online Resources , 2011 .

[65]  C. C. Robinson,et al.  New Benchmarks in Higher Education: Student Engagement in Online Learning , 2008 .

[66]  D. Randy Garrison,et al.  The first decade of the community of inquiry framework: A retrospective , 2010, Internet High. Educ..

[67]  Anne C. Frenzel,et al.  Measuring emotions in students learning and performance: The Achievement Emotions Questionnaire (AE , 2011 .

[68]  Kristin E. Voelkl,et al.  Disruptive and Inattentive-Withdrawn Behavior and Achievement among Fourth Graders , 1995, The Elementary School Journal.

[69]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .

[70]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[71]  A. L. Edwards The social desirability variable in personality assessment and research , 1958 .

[72]  Chris Piech,et al.  Deconstructing disengagement: analyzing learner subpopulations in massive open online courses , 2013, LAK '13.

[73]  Philip H. Winne,et al.  Measuring Self-Regulated Learning , 2000 .

[74]  Bernard J. Jansen,et al.  Analyzing MOOC discussion forum messages to identify cognitive learning information exchanges , 2015, ASIST.

[75]  Carlos Delgado Kloos,et al.  GLASS: a learning analytics visualization tool , 2012, LAK '12.

[76]  Bart Rienties,et al.  Linking students' timing of engagement to learning design and academic performance , 2018, LAK.

[77]  Alyssa Friend Wise,et al.  Learning analytics implementation design , 2017 .

[78]  Erik Duval,et al.  Goal-oriented visualizations of activity tracking: a case study with engineering students , 2012, LAK.

[79]  Maren Scheffel,et al.  License to evaluate: preparing learning analytics dashboards for educational practice , 2018, LAK.

[80]  D. Keegan,et al.  Distance education : new perspectives , 2013 .

[81]  Philip H. Winne,et al.  Exploring Individual Differences in Studying Strategies Using Graph Theoretic Statistics. , 1994 .

[82]  Eunjoon Rachel Um,et al.  Emotional design in multimedia learning. , 2012 .

[83]  Peter Brusilovsky,et al.  An Intelligent Interface for Learning Content: Combining an Open Learner Model and Social Comparison to Support Self-Regulated Learning and Engagement , 2016, IUI.

[84]  Sara McNeil,et al.  Students' patterns of engagement and course performance in a Massive Open Online Course , 2016, Comput. Educ..

[85]  Kasia Muldner,et al.  Exploring the Impact of a Learning Dashboard on Student Affect , 2015, AIED.

[86]  Minjuan Wang,et al.  Affective e-Learning: Using "Emotional" Data to Improve Learning in Pervasive Learning Environment , 2009, J. Educ. Technol. Soc..

[87]  H. M. Marks,et al.  Student Engagement in Instructional Activity: Patterns in the Elementary, Middle, and High School Years , 2000 .

[88]  Philip H. Winne,et al.  Minimizing the black box problem to enhance the validity of theories about instructional effects , 1982 .

[89]  Christoph Rensing,et al.  PeerLA - Assistant for Individual Learning Goals and Self-Regulation Competency Improvement in Online Learning Scenarios , 2016, 2016 IEEE 16th International Conference on Advanced Learning Technologies (ICALT).

[90]  D. Entwisle,et al.  From First Grade Forward: Early Foundations of High School Dropout. , 1997 .

[91]  Hendrik Drachsler,et al.  The pulse of learning analytics understandings and expectations from the stakeholders , 2012, LAK.

[92]  L. Morris,et al.  Tracking student behavior, persistence, and achievement in online courses , 2005, Internet High. Educ..

[93]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[94]  Kimberly E. Arnold Signals: Applying Academic Analytics. , 2010 .

[95]  Carlos Delgado Kloos,et al.  Delving into Participants’ Profiles and Use of Social Tools in MOOCs , 2014, IEEE Transactions on Learning Technologies.

[96]  Lise Getoor,et al.  Uncovering hidden engagement patterns for predicting learner performance in MOOCs , 2014, L@S.

[97]  Janet L. Lear,et al.  Interactivity/Community Process Model for the Online Education Environment , 2010 .

[98]  George D. Kuh,et al.  Student Engagement and Student Learning: Testing the Linkages* , 2006 .

[99]  Marco Kalz,et al.  Time will tell: The role of mobile learning analytics in self-regulated learning , 2015, Comput. Educ..

[100]  Marek Hatala,et al.  Associations Between Students' Approaches to Learning and Learning Analytics Visualizations , 2016, LAL@LAK.

[101]  Reinhard Pekrun,et al.  A Social-Cognitive, Control-Value Theory of Achievement Emotions , 2000 .

[102]  Po-Yao Chao,et al.  Open Student Models of Core Competencies at the Curriculum Level: Using Learning Analytics for Student Reflection , 2017, IEEE Transactions on Emerging Topics in Computing.

[103]  Erik Duval,et al.  The student activity meter for awareness and self-reflection , 2012, CHI Extended Abstracts.

[104]  James E. Willis,et al.  Analytics Through an Institutional Lens: Definition, Theory, Design, and Impact , 2014 .

[105]  George S. Howard,et al.  Response-shift bias: A source of contamination of self-report measures. , 1979 .

[106]  Ji Won You,et al.  Examining the Effect of Academic Procrastination on Achievement Using LMS Data in e-Learning , 2015, J. Educ. Technol. Soc..

[107]  P. Peterson,et al.  Students’ Cognitions and Time on Task During Mathematics Instruction , 1984 .

[108]  Yuhei Yamauchi,et al.  iTree: does the mobile phone encourage learners to be more involved in collaborative learning? , 2005, CSCL.

[109]  George Siemens,et al.  Let’s not forget: Learning analytics are about learning , 2015 .

[110]  Erik Duval,et al.  Learning Analytics Dashboard Applications , 2013 .

[111]  J. Hattie,et al.  The Power of Feedback , 2007 .

[112]  Arthur Bangert,et al.  Identifying factors underlying the quality of online teaching effectiveness: An exploratory study , 2006, J. Comput. High. Educ..

[113]  Anastasios A. Economides,et al.  Learning Analytics and Educational Data Mining in Practice: A Systematic Literature Review of Empirical Evidence , 2014, J. Educ. Technol. Soc..

[114]  Marcia D. Dixson Measuring Student Engagement in the Online Course: The Online Student Engagement Scale (OSE). , 2015 .

[115]  Ryan S. Baker,et al.  Educational Data Mining and Learning Analytics , 2014 .

[116]  Marek Hatala,et al.  A qualitative evaluation of evolution of a learning analytics tool , 2012, Comput. Educ..

[117]  S. Christenson,et al.  Handbook of Research on Student Engagement , 2012 .

[118]  K. Robert Lai,et al.  Who will pass? Analyzing learner behaviors in MOOCs , 2016, Res. Pract. Technol. Enhanc. Learn..

[119]  Peter Brusilovsky,et al.  Mastery Grids: An Open Source Social Educational Progress Visualization , 2014, EC-TEL.

[120]  George D. Kuh The national survey of student engagement: Conceptual and empirical foundations , 2009 .

[121]  E. Skinner,et al.  Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year. , 1993 .

[122]  Julie A. Gray,et al.  The Effects of Student Engagement, Student Satisfaction, and Perceived Learning in Online Learning Environments , 2016 .

[123]  Tim Urdan,et al.  Academic Self-Handicapping: What We Know, What More There is to Learn , 2001 .

[124]  Eugene W. Lyman,et al.  An Instructional Design Framework for Fostering Student Engagement in Online Learning Environments , 2016 .

[125]  P. Winne,et al.  Feedback and Self-Regulated Learning: A Theoretical Synthesis , 1995 .

[126]  Abdullah Karaksha,et al.  Student Engagement in Pharmacology Courses Using Online Learning Tools , 2013, American Journal of Pharmaceutical Education.

[127]  D. Mcloughlin,et al.  An analysis of higher order thinking in online discussions , 2009 .

[128]  William R. Wiencke,et al.  EXPLORING THE INTERACTION EQUATION: VALIDATING A RUBRIC TO ASSESS AND ENCOURAGE INTERACTION IN DISTANCE COURSES , 2019, Online Learning.

[129]  Hendrik Drachsler,et al.  Translating Learning into Numbers: A Generic Framework for Learning Analytics , 2012, J. Educ. Technol. Soc..

[130]  James J. Appleton,et al.  Student engagement with school: Critical conceptual and methodological issues of the construct. , 2008 .

[131]  Rebecca Ferguson,et al.  Learning analytics: drivers, developments and challenges , 2012 .

[132]  John C. Nesbit,et al.  Designs for learning analytics to support information problem solving , 2017 .

[133]  Katrien Verbert,et al.  Trends and issues in student-facing learning analytics reporting systems research , 2017, LAK.

[134]  B. Zimmerman,et al.  Handbook of Self-Regulation of Learning and Performance , 2011 .

[135]  Karen Swan,et al.  Virtual interaction: Design factors affecting student satisfaction and perceived learning in asynchronous online courses , 2001 .

[136]  Johann Ari Larusson,et al.  Learning Analytics: From Research to Practice , 2014 .

[137]  R K Deppe,et al.  Self-handicapping and intrinsic motivation: buffering intrinsic motivation from the threat of failure. , 1996, Journal of personality and social psychology.

[138]  H. Coates Attracting, engaging and retaining : new conversations about learning , 2008 .

[139]  V. Shute Focus on Formative Feedback , 2008 .

[140]  S. Kuhar,et al.  Taking Advantage of Education Data: Advanced Data Analysis and Reporting in Virtual Learning Environments , 2011 .

[141]  Barbara Lockee,et al.  Building modern online social presence: A review of social presence theory and its instructional design implications for future trends , 2012, Education and Information Technologies.

[142]  Alyssa Friend Wise,et al.  Developing Learning Analytics Design Knowledge in the "Middle Space": The Student Tuning Model and Align Design Framework for Learning Analytics Use , 2016 .

[143]  Davide Marengo,et al.  Sharing feelings online: studying emotional well-being via automated text analysis of Facebook posts , 2015, Front. Psychol..

[144]  John P. Campbell,et al.  Academic Analytics: A New Tool for a New Era. , 2007 .

[145]  R. Azevedo Defining and Measuring Engagement and Learning in Science: Conceptual, Theoretical, Methodological, and Analytical Issues , 2015 .

[146]  Anna-Katharina Praetorius,et al.  The Effects of Teachers' Reference Norm Orientations on Students' Implicit Theories and Academic Self-Concepts , 2017 .