How do we start? State and Directions of

The analysis of data collected from user interactions with educational and information technology has attracted much attention as a promising approach to advancing our understanding of the learning process. This promise motivated the emergence of the field of learning analytics and pushed the education sector towards increased application of data for strategic decision-making. This paper addresses a commonly posed question asked by educators, managers, and administrators embarking on learning analytics in higher education – how do we start institutional learning analytics adoption? The paper first defines learning analytics and touches on lessons learned from some well-known case studies. The paper then reviews the current state of institutional adoption of learning analytics by examining evidence produced in several studies conducted worldwide. The paper next outlines directions for learning analytics adoption that should enable for a system-wide institutional transformation. The paper concludes with a summary of critical challenges that require attention in order for learning analytics to make a long-term impact on research and practice of learning and teaching. The paper emphasizes that learning analytics cannot be reduced to a simplistic rhetoric of quick technological fixes. Rather, learning analytics advocates for holistic approaches that account for and support complexities associated with specific characteristics of different educational systems and institutions.

[1]  Martin Ebner,et al.  De-Identification in Learning Analytics , 2016, J. Learn. Anal..

[2]  D. Garrison,et al.  Researching the community of inquiry framework: Review, issues, and future directions , 2007, Internet High. Educ..

[3]  George Siemens,et al.  Current state and future trends: a citation network analysis of the learning analytics field , 2014, LAK.

[4]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[5]  Simon A. Fraser.  Does Time-on-task Estimation Matter? Implications for the Validity of Learning Analytics Findings , 2016 .

[6]  Dragan Gasevic,et al.  Learning Analytics and the Imperative for Theory Driven Research , 2016 .

[7]  Marek Hatala,et al.  A qualitative evaluation of evolution of a learning analytics tool , 2012, Comput. Educ..

[8]  Chris Rust,et al.  A scholarly approach to solving the feedback dilemma in practice , 2016 .

[9]  Masters Ao,et al.  Challenging our most able students , 2015 .

[10]  Maria Bannert,et al.  e-Research and learning theory: What do sequence and process mining methods contribute? , 2014, Br. J. Educ. Technol..

[11]  Linda Corrin,et al.  Student retention and learning analytics: A snapshot of Australian practices and a framework for advancement , 2015 .

[12]  J. Manyika.  Big data: The next frontier for innovation, competition, and productivity , 2011 .

[13]  John Fritz,et al.  Classroom walls that talk: Using online course activity data of successful students to raise self-awareness of underperforming peers , 2011, Internet High. Educ..

[14]  Rebecca Ferguson,et al.  Setting learning analytics in context: overcoming the barriers to large-scale adoption , 2014, LAK '14.

[15]  Kate Miller,et al.  Better Than Expected: Using Learning Analytics to Promote Student Success in Gateway Science , 2014 .

[16]  Michael Prosser,et al.  Improving the quality of student learning: the influence of learning context and student approaches to learning on learning outcomes , 1991 .

[17]  Niall Sclater,et al.  Code of practice for learning analytics , 2015 .

[18]  Dragan Gasevic,et al.  Ethics and Privacy as Enablers of Learning Analytics , 2016 .

[19]  Matthew D. Pistilli,et al.  Course signals at Purdue: using learning analytics to increase student success , 2012, LAK.

[20]  H. Johnson,et al.  Defunding Higher Education What Are the Effects on College Enrollment , 2012 .

[21]  Justin Reich,et al.  Socioeconomic status and MOOC enrollment: enriching demographic information with external datasets , 2015, LAK.

[22]  Abigail Selzer King,et al.  Using Signals for appropriate feedback: Perceptions and practices , 2011, Comput. Educ..

[23]  Marek Hatala,et al.  The role of achievement goal orientations when studying effect of learning analytics visualizations , 2016, LAK.

[24]  E. Deci,et al.  The effects of instructors' autonomy support and students' autonomous motivation on learning organic chemistry: A self-determination theory perspective , 2000 .

[25]  Shane Dawson,et al.  Informing Pedagogical Action , 2013 .

[26]  Shane Dawson,et al.  Mining LMS data to develop an "early warning system" for educators: A proof of concept , 2010, Comput. Educ..

[27]  George Siemens,et al.  Preparing for the Digital University: A Review of the History and Current State of Distance, Blended and Online Learning , 2015 .

[28]  Shane Dawson,et al.  Numbers Are Not Enough. Why e-Learning Analytics Failed to Inform an Institutional Strategic Plan , 2012, J. Educ. Technol. Soc..

[29]  Jeroen Janssen,et al.  Group awareness tools: It's what you do with it that matters , 2011, Comput. Hum. Behav..

[30]  Carlos Delgado Kloos,et al.  Monitoring student progress using virtual appliances: A case study , 2012, Comput. Educ..

[31]  Ronald Yanosky.  Institutional Data Management in Higher Education. ECAR Key Findings. , 2009 .

[32]  Geraldine Clarebout,et al.  Regulation of tool-use within a blended course: Student differences and performance effects , 2013, Comput. Educ..

[33]  John Patrick Campbell.  Utilizing student data within the course management system to determine undergraduate student academic success: An exploratory study , 2007 .

[34]  Zdenek Zdráhal,et al.  Data literacy for learning analytics , 2016, LAK.

[35]  D. Boud,et al.  Rethinking models of feedback for learning: the challenge of design , 2013 .

[36]  Ryan Shaun Joazeiro de Baker,et al.  Off-task behavior in the cognitive tutor classroom: when students "game the system" , 2004, CHI.

[37]  Simon Buckingham Shum,et al.  Learning dispositions and transferable competencies: pedagogy, modelling and learning analytics , 2012, International Conference on Learning Analytics and Knowledge.

[38]  Philip H. Winne,et al.  Studying as self-regulated learning. , 1998 .

[39]  Ido Roll,et al.  Understanding, evaluating, and supporting self-regulated learning using learning analytics , 2015, J. Learn. Anal..

[40]  Karen Littleton,et al.  Discourse Centric Learning Analytics: Mapping the Terrain , 2015, J. Learn. Anal..

[41]  Lewis Elton,et al.  Goodhart's Law and Performance Indicators in Higher Education , 2004 .

[42]  George Siemens,et al.  Let’s not forget: Learning analytics are about learning , 2015 .

[43]  Shane Dawson,et al.  A Study of the Relationship between Student Social Networks and Sense of Community , 2008, J. Educ. Technol. Soc..

[44]  C. Phillips,et al.  The use of flipped classrooms in higher education: A scoping review , 2015, Internet High. Educ..

[45]  Albert T. Corbett,et al.  Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments , 2008 .

[46]  Hendrik Drachsler,et al.  Privacy and analytics: it's a DELICATE issue a checklist for trusted learning analytics , 2016, LAK.

[47]  Geraldine Clarebout,et al.  Tool-use in a blended undergraduate course: In Search of user profiles , 2011, Comput. Educ..

[48]  Dragan Gasevic,et al.  Learning analytics should not promote one size fits all: The effects of instructional conditions in predicting academic success , 2016, Internet High. Educ..

[49]  Alyssa Friend Wise,et al.  Designing pedagogical interventions to support student use of learning analytics , 2014, LAK.

[50]  Marek Hatala,et al.  Penetrating the black box of time-on-task estimation , 2015, LAK.

[51]  D. Gašević,et al.  “Choose Your Classmates, Your GPA Is at Stake!” , 2013 .

[52]  Dominic Barton,et al.  Making advanced analytics work for you. , 2012, Harvard business review.

[53]  C. Abraham,et al.  Psychological correlates of university students' academic performance: a systematic review and meta-analysis. , 2012, Psychological bulletin.

[54]  Rebecca Ferguson,et al.  Guest Editorial: Ethics and Privacy in Learning Analytics , 2016, J. Learn. Anal..

[55]  Dragan Gasevic,et al.  Translating network position into performance: importance of centrality in different network configurations , 2016, LAK.

[56]  Steven Lonn,et al.  A Learning Management System-Based Early Warning System for Academic Advising in Undergraduate Engineering , 2014 .

[57]  K. Trigwell,et al.  Relations between teachers' approaches to teaching and students' approaches to learning , 1999 .

[58]  Josh Jarrett.  Bigfoot, Goldilocks, and Moonshots. A Report from the Frontiers of Personalized Learning. , 2013 .

[59]  Simon Buckingham Shum,et al.  Learning Analytics for 21st Century Competencies , 2016, J. Learn. Anal..

[60]  Marek Hatala,et al.  Learning at distance: Effects of interaction traces on academic achievement , 2015, Comput. Educ..

[61]  Raymond T. Sparrowe,et al.  Social Networks and the Performance of Individuals and Groups , 2001 .

[62]  Michelle K. Smith,et al.  Active learning increases student performance in science, engineering, and mathematics , 2014, Proceedings of the National Academy of Sciences.

[63]  George Siemens,et al.  Guest Editorial - Learning and Knowledge Analytics , 2012, J. Educ. Technol. Soc..

[64]  Barbara Wasson,et al.  Data Literacy and Use for Teaching , 2015 .

[65]  Learning Analytics: Moving from Concept to Practice , 2012 .

[66]  Larry Ambrose,et al.  The power of feedback. , 2002, Healthcare executive.

[67]  M. Moore.  Editorial: Three types of interaction , 1989 .

[68]  Marek Hatala,et al.  Analytics of communities of inquiry: Effects of learning technology use on cognitive presence in asynchronous online discussions , 2015, Internet High. Educ..

[69]  Erik Duval,et al.  Learning Analytics Dashboard Applications , 2013 .

[70]  Leah P. Macfadyen,et al.  Embracing Big Data in Complex Educational Systems: The Learning Analytics Imperative and the Policy Challenge. , 2014 .

[71]  S. Dawson.  SNAPP : Realising the affordances of real-time SNA within networked learning environments , 2010 .

[72]  Tim McKay,et al.  What to do with actionable intelligence: E2Coach as an intervention engine , 2012, LAK '12.

[73]  Iris Vessey,et al.  Cognitive Fit: A Theory‐Based Analysis of the Graphs Versus Tables Literature* , 1991 .

[74]  George Siemens,et al.  Learning Analytics , 2013 .

[75]  Paul Prinsloo,et al.  Student Vulnerability, Agency and Learning Analytics: An Exploration , 2016, J. Learn. Anal..

[76]  Philip C. Abrami,et al.  Media and Pedagogy in Undergraduate Distance Education: A Theory-Based Meta-Analysis of Empirical Literature , 2006 .

[77]  Shane Dawson,et al.  Measuring creative potential: Using social network analysis to monitor a learners' creative capacity , 2011 .

[78]  Nancy Sherman Shapiro,et al.  Learning Communities: Reforming Undergraduate Education , 2006 .

[79]  A. Wise,et al.  Why Theory Matters More than Ever in the Age of Big Data , 2015, J. Learn. Anal..