Making sense of learner and learning Big Data: reviewing five years of Data Wrangling at the Open University UK

ABSTRACT Most distance learning institutions collect vast amounts of learning data. Making sense of this ‘Big Data’ can be a challenge, in particular when data are stored at different data warehouses and require advanced statistical skills to interpret complex patterns of data. As a leading institute on learning analytics, the Open University UK instigated in 2012 a Data Wrangling initiative. This provided every Faculty with a dedicated academic with expertise data analysis and whose task is to provide strategic, pedagogical and sense-making advice to staff and senior management. Given substantial changes within the OU (e.g. new Faculty structure, real-time dashboards, two large-scale adoptions of predictive analytics approaches, increased reliance on analytics), this embedded case study provides an in-depth review of lessons learned of five years of data wrangling. We will elaborate on the design of the new structure, its strengths and potential weaknesses, and affordances to be adopted by other institutions.

[1]  J. B. Arbaugh,et al.  System, scholar or students? Which most influences online MBA course effectiveness? , 2014, J. Comput. Assist. Learn..

[2]  Glenn A. Bowen.  Document Analysis as a Qualitative Research Method , 2009 .

[3]  Alison Ashby.  Monitoring student retention in the Open University: definition, measurement, interpretation and action , 2004 .

[4]  Marilyn Lichtman,et al.  Qualitative Research in Education: A User's Guide , 2006 .

[5]  Carolyn Penstein Rosé,et al.  Learning analytics and machine learning , 2014, LAK.

[6]  David Feinleib.  Bricks to Clicks , 2017 .

[7]  B. Rienties,et al.  Online learning experiences of new versus continuing learners: a large-scale replication study , 2017 .

[8]  Peter Mork,et al.  From Data to Decisions: A Value Chain for Big Data , 2013, IT Professional.

[9]  Bart Rienties,et al.  The impact of learning design on student behaviour, satisfaction and performance: A cross-institutional comparison across 151 modules , 2016, Comput. Hum. Behav..

[10]  Anastasios A. Economides,et al.  Learning Analytics for Smart Learning Environments: A Meta-Analysis of Empirical Research Results from 2009 to 2015 , 2016 .

[11]  John T. E. Richardson,et al.  Investigating the relationship between variations in students' perceptions of their academic environment and variations in study behaviour in distance education. , 2006, The British journal of educational psychology.

[12]  Rebecca Ferguson,et al.  Innovating Pedagogy 2016: Open University Innovation Report 5 , 2016 .

[13]  Doug Clow.  Data wranglers: human interpreters to help close the feedback loop , 2014, LAK '14.

[14]  William A. Reitzel.  The Strategic Plan , 1956 .

[15]  Erik Duval,et al.  Dataset-driven research for improving recommender systems for learning , 2011, LAK.

[16]  Martin Weller,et al.  Final Project Report of the OULDI- JISC Project: Challenge and Change in Curriculum Design Process, Communities, Visualisation and Practice , 2012 .

[17]  Dirk T. Tempelaar,et al.  In search for the most informative data for feedback generation: Learning analytics in a data-rich context , 2015, Comput. Hum. Behav..

[18]  Zdenek Zdráhal,et al.  Improving retention: predicting at-risk students by analysing clicking behaviour in a virtual learning environment , 2013, LAK '13.

[19]  Bart Rienties,et al.  Analysing 157 learning designs using learning analytic approaches as a means to evaluate the impact of pedagogical decision making , 2016, Br. J. Educ. Technol..

[20]  John T. E. Richardson,et al.  The attainment of White and ethnic minority students in distance education , 2012 .

[21]  B. Rienties,et al.  Bridge building potential in cross-cultural learning: a mixed method study , 2015 .

[22]  Carol Calvert.  Developing a model and applications for probabilities of student success: a case study of predictive analytics , 2014 .

[23]  Rebecca Ferguson,et al.  Research Evidence on the Use of Learning Analytics: Implications for Education Policy , 2016 .

[24]  Bart Rienties,et al.  Learning Design – creative design to visualise learning activities , 2016 .

[25]  Bart Rienties,et al.  Implementing a Learning Analytics Intervention and Evaluation Framework: what works? , 2017 .

[26]  V. J. Connors,et al.  Strategic plan. , 1999, Journal of the American Optometric Association.

[27]  Bart Rienties,et al.  Scholarly insight 2016: a Data wrangler perspective , 2016 .

[28]  Roy D. Pea,et al.  Educational data scientists: a scarce breed , 2013, LAK '13.

[29]  Marie Bienkowski,et al.  Enhancing Teaching and Learning Through Educational Data Mining and Learning Analytics: An Issue Brief , 2012 .

[30]  Gráinne Conole,et al.  Designing for Learning in an Open World , 2012 .

[31]  Bart Rienties,et al.  Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK , 2016 .

[32]  Martin Hlosta,et al.  OU Analyse: analysing at-risk students at The Open University , 2015 .

[33]  Tom Inkelaar,et al.  Challenging the ‘distance education deficit’ through ‘motivational emails’ , 2015 .

[34]  Doug Clow.  An overview of learning analytics , 2013 .

[35]  Rebecca Ferguson,et al.  Where is the evidence?: a call to action for learning analytics , 2017, LAK.

[36]  Hendrik Drachsler,et al.  Privacy and Analytics – it’s a DELICATE issue. A Checklist to establish trusted Learning Analytics , 2016 .

[37]  Leah P. Macfadyen,et al.  Embracing Big Data in Complex Educational Systems: The Learning Analytics Imperative and the Policy Challenge. , 2014 .

[38]  R. Yin.  Case Study Research: Design and Methods , 1984 .

[39]  Rebecca Ferguson,et al.  Innovating Pedagogy 2014 , 2014 .

[40]  Peter Twining,et al.  Some guidance on conducting and reporting qualitative studies , 2017, Comput. Educ..

[41]  O. Simpson.  The impact on retention of interventions to support distance learning students , 2004 .