Learner Dashboards a Double-Edged Sword? Students' Sense-Making of a Collaborative Critical Reading and Learning Analytics Environment for Fostering 21st Century Literacies

The affordances of learning analytics (LA) tools and solutions are being increasingly harnessed for enhancing 21st century pedagogical and learning strategies and outcomes. However, use cases and empirical understandings of students’ experiences with LA tools and environments aimed at fostering 21st century literacies, especially in the K-12 schooling sector and in Asian education contexts remain relatively scarce in the field. Our paper addresses this knowledge gap in two ways. First, we present a first iteration design of a computer-supported collaborative critical reading and LA environment, WiREAD, and its 16-week implementation in a Singapore high school. Second, we foreground students’ evaluative accounts of the benefits and drawbacks associated with this techno-pedagogical innovation. Our analysis of students’ collective sense-making pointed to a number of potentialities and perils associated with the design and use of LA dashboards. Positives included (1) fostering greater self-awareness, reflective and self-regulatory learning dispositions, (2) enhancing learning motivation and engagement, and (3) nurturing connective literacy among students. The motivational value of peer-referenced LA visualisations for stimulating healthy competition and game-like learning was identified, alongside the perils of these serving to demoralise, pressurise and trigger complacency in learners. By focusing on students’ experiences and interpretations of how the LA dashboard visualizations impacted their learning motivation and outcomes, this paper aims to shed insights into the pedagogical complexities of designing LA that considers the voices of learners as a critical stakeholder group.

[1]  Alyssa Friend Wise,et al.  Developing Learning Analytics Design Knowledge in the "Middle Space": The Student Tuning Model and Align Design Framework for Learning Analytics Use , 2016 .

[2]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[3]  Erik Duval,et al.  Learning Analytics Dashboard Applications , 2013 .

[4]  Karen Littleton,et al.  Discourse Centric Learning Analytics: Mapping the Terrain , 2015, J. Learn. Anal..

[5]  D. Wiliam What Is Assessment for Learning , 2011 .

[6]  H. Levin More than just test scores , 2012 .

[7]  Marylène Gagné Self-Determination Theory in the Work Domain , 2014 .

[8]  Carolyn Penstein Rosé,et al.  Analyzing collaborative learning processes automatically: Exploiting the advances of computational linguistics in computer-supported collaborative learning , 2008, Int. J. Comput. Support. Collab. Learn..

[9]  Terry Ryan,et al.  Enhancing student knowledge acquisition from online learning conversations , 2013, International Journal of Computer-Supported Collaborative Learning.

[10]  V. Shute Focus on Formative Feedback , 2008 .

[11]  Carolyn Penstein Rosé,et al.  Technology Support for Discussion Based Learning: From Computer Supported Collaborative Learning to the Future of Massive Open Online Courses , 2016, International Journal of Artificial Intelligence in Education.

[12]  Wendy McColskey,et al.  Differential effects of norm-referenced and self-referenced feedback on performance expectancies, attributions, and motivation , 1985 .

[13]  A. Kluger,et al.  The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. , 1996 .

[14]  James Paul Gee,et al.  Game–like learning: An example of situated learning and implications for opportunity to learn , 2008 .

[15]  Rebecca Ferguson,et al.  Social learning analytics: five approaches , 2012, LAK.

[16]  Daniel C. Moos,et al.  Why is externally-facilitated regulated learning more effective than self-regulated learning with hypermedia? , 2008 .

[17]  J. Tan,et al.  From Literacy to Multiliteracies: Diverse Learners and Pedagogical Practice , 2009 .

[18]  R. Yin,et al.  Case Study Research: Design and Methods (4th ed. , 2009 .

[19]  B. Street Social Literacies: Critical Approaches to Literacy in Development, Ethnography and Education , 1995 .

[20]  Antero Garcia,et al.  The Council of Youth Research: Critical Literacy and Civic Agency in the Digital Age , 2015 .

[21]  J. Tan Digital kids, analogue students : a mixed methods study of students' engagement with a school-based Web 2.0 learning innovation , 2009 .

[22]  I. N. A. C. I. J. H. Fowler Book Review: Connected: The surprising power of our social networks and how they shape our lives. , 2009 .

[23]  María Jesús Rodríguez-Triana,et al.  Understanding learning at a glance: an overview of learning dashboard studies , 2016, LAK.

[24]  Erik Duval,et al.  Analysis and Reflections on the Third Learning Analytics and Knowledge Conference (LAK 2013) , 2014, J. Learn. Anal..

[25]  Philip H. Winne,et al.  nStudy: An Internet Tool to Support Learning, Collaboration and Researching Learning Strategies , 2009 .

[26]  Shui-fong Lam,et al.  Effects of different evaluative feedback on students’ self-efficacy in learning , 2010 .

[27]  J. Gee Socio-Cultural Approaches to Literacy (Literacies) , 1991, Annual Review of Applied Linguistics.

[28]  B. Street What's 'new' in New Literacy Studies? Critical approaches to literacy in theory and practice , 2003 .

[29]  George Siemens,et al.  Analytics to literacies: The development of a learning analytics framework for multiliteracies assessment , 2014 .

[30]  Mary Kalantzis,et al.  A Pedagogy of Multiliteracies: Learning by Design , 2015 .

[31]  Maureen Walsh,et al.  Multimodal literacy: What does it mean for classroom practice? , 2010, The Australian Journal of Language and Literacy.

[32]  S. Michaels,et al.  A pedagogy of Multiliteracies Designing Social Futures , 1996 .

[33]  R. Stiggins From Formative Assessment to Assessment for Learning: A Path to Success in Standards-Based Schools , 2005 .

[34]  Dirk T. Tempelaar,et al.  In search for the most informative data for feedback generation: Learning analytics in a data-rich context , 2015, Comput. Hum. Behav..

[35]  R. Paul,et al.  Critical Thinking: Tools for Taking Charge of Your Learning and Your Life , 2011 .

[36]  Keith Turvey,et al.  Towards deeper learning through creativity within online communities in primary education , 2006, Comput. Educ..

[37]  Alyssa Friend Wise,et al.  Learning Analytics for Online Discussions: Embedded and Extracted Approaches , 2014, J. Learn. Anal..

[38]  Alyssa Friend Wise,et al.  Designing pedagogical interventions to support student use of learning analytics , 2014, LAK.

[39]  Shane Dawson,et al.  Informing Pedagogical Action , 2013 .

[40]  Marek Hatala,et al.  The role of achievement goal orientations when studying effect of learning analytics visualizations , 2016, LAK.

[41]  Ming-Te Wang,et al.  Adolescent Behavioral, Emotional, and Cognitive Engagement Trajectories in School and Their Differential Relations to Educational Success. , 2012 .

[42]  Erik Duval,et al.  The student activity meter for awareness and self-reflection , 2012, CHI Extended Abstracts.

[43]  Matthew B. Miles,et al.  Qualitative Data Analysis: An Expanded Sourcebook , 1994 .

[44]  J. Tan A DIALOGIC FRAMEWORK FOR ASSESSING COLLECTIVE CREATIVITY IN COMPUTER-SUPPORTED COLLABORATIVE PROBLEM-SOLVING TASKS , 2014 .

[45]  Rebecca Ferguson,et al.  Social Learning Analytics , 2012, J. Educ. Technol. Soc..

[46]  Andrew J. Martin,et al.  The role of personal best (PB) and dichotomous achievement goals in students’ academic motivation and engagement: a longitudinal investigation , 2016 .

[47]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..