Measuring and Visualizing Students’ Behavioral Engagement in Writing Activities

Engagement is critical to the success of learning activities such as writing, and can be promoted with appropriate feedback. Current engagement measures rely mostly on data collected by observers or self-reported by the participants. In this paper, we describe a learning analytic system called Tracer, which derives behavioral engagement measures and creates visualizations of behavioral patterns of students writing on a cloud-based application. The tool records the intermediate stages of document development and uses this data to measure learners' behavioral engagement and derive three visualizations. Writers (N= 23 University students) participated in a controlled one-hour writing session in which they post-facto self-reported their level of behavioral engagement. Results show that the level of behavioral engagement automatically estimated by the system correlates with the level reported by the participants. Additionally, users stated that the visualizations were coherent with their writing activity and were useful to help them reflect on the writing process.

[1]  Shane Dawson,et al.  SNAPP: a bird's-eye view of temporal participant interaction , 2011, LAK.

[2]  Rebecca Ferguson,et al.  Learning analytics to identify exploratory dialogue within synchronous text chat , 2011, LAK.

[3]  Ming Liu,et al.  Tracer: A Tool to Measure and Visualize Student Engagement in Writing Activities , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[4]  Abigail Selzer King,et al.  Using Signals for appropriate feedback: Perceptions and practices , 2011, Comput. Educ..

[5]  Rebecca Ferguson,et al.  An evaluation of learning analytics to identify exploratory dialogue in online discussions , 2013, LAK '13.

[6]  Kalina Yacef,et al.  Collaborative Writing Support Tools on the Cloud , 2011, IEEE Transactions on Learning Technologies.

[7]  Kevin C. Almeroth,et al.  Measuring Learner Engagement in Computer-Equipped College Classrooms , 2008 .

[8]  Marcus Specht,et al.  A first approach to “Learning Dashboards” in formal learning contexts , 2011 .

[9]  Jennifer A. Fredricks,et al.  School Engagement: Potential of the Concept, State of the Evidence , 2004 .

[10]  Steven Lonn,et al.  Bridging the gap from knowledge to action: putting analytics in the hands of academic advisors , 2012, LAK '12.

[11]  Andrew J. Martin Examining a multidimensional model of student motivation and engagement using a construct validation approach. , 2007, The British journal of educational psychology.

[12]  Valentin Zacharias,et al.  Live interest meter: learning from quantified feedback in mass lectures , 2013, LAK '13.

[13]  Erik Duval,et al.  Learning Analytics Dashboard Applications , 2013 .

[14]  Lorena Blasco-Arcas,et al.  Using clickers in class. The role of interactivity, active collaborative learning and engagement in learning performance , 2013, Comput. Educ..

[15]  John P. Campbell,et al.  Academic Analytics: A New Tool for a New Era. , 2007 .

[16]  Kalina Yacef,et al.  Analysis of collaborative writing processes using revision maps and probabilistic topic models , 2013, LAK '13.

[17]  Muhammad M. Abdel Latif,et al.  A State-of-the-Art Review of the Real-Time Computer-Aided Study of the Writing Process , 2008 .

[18]  Tim McKay,et al.  What to do with actionable intelligence: E2Coach as an intervention engine , 2012, LAK '12.

[19]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[20]  Rebecca Ferguson,et al.  Social learning analytics: five approaches , 2012, LAK.

[21]  Sidney K. D'Mello,et al.  Detecting boredom and engagement during writing with keystroke analysis, task appraisals, and stable traits , 2013, IUI '13.

[22]  Vania Dimitrova,et al.  CourseVis: A graphical student monitoring tool for supporting instructors in web-based distance courses , 2007, Int. J. Hum. Comput. Stud..

[23]  Symeon Retalis,et al.  Utilising a collaborative macro-script to enhance student engagement: A mixed method study in a 3D virtual environment , 2012, Comput. Educ..

[24]  Melissa Cole,et al.  Using Wiki technology to support student engagement: Lessons from the trenches , 2009, Comput. Educ..

[25]  Carlos Delgado Kloos,et al.  Provision of awareness of learners' emotions through visualizations in a computer interaction-based environment , 2013, Expert Syst. Appl..

[26]  John Fritz,et al.  Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers , 2011, Internet High. Educ..

[27]  Kevin R. Guidry,et al.  Engaging online learners: The impact of Web-based learning technology on college student engagement , 2010, Comput. Educ..

[28]  Rafael A. Calvo,et al.  Frontiers of Affect-Aware Learning Technologies , 2012, IEEE Intelligent Systems.

[29]  Learning Analytics: Moving from Concept to Practice , 2012 .

[30]  Hanan Ayad,et al.  Improving student success using predictive models and data visualisations , 2012 .

[31]  Kennon M. Sheldon,et al.  Standards, Accountability, and School Reform: Perils and Pitfalls , 1998, Teachers College Record: The Voice of Scholarship in Education.