Exploratory Analysis in Learning Analytics

This article summarizes the methods, observations, challenges and implications for exploratory analysis drawn from two learning analytics research projects. The cases include an analysis of a games-based virtual performance assessment and an analysis of data from 52,000 students over a 5-year period at a large Australian university. The complex datasets were analyzed and iteratively modeled with a variety of computationally intensive methods to provide the most effective outcomes for learning assessment, performance management and learner tracking. The article presents the research contexts, the tools and methods used in the exploratory phases of analysis, the major findings and the implications for learning analytics research methods.

[1]  Chris Dede,et al.  Affordances and Limitations of Immersive Participatory Augmented Reality Simulations for Teaching and Learning , 2009 .

[2]  David Gibson,et al.  Game Changers for Transforming Learning Environments , 2012 .

[3]  L. Amaral,et al.  Duality between Time Series and Networks , 2011, PloS one.

[4]  Haruna Chiroma,et al.  Data Mining for Education Decision Support: A Review , 2014, Int. J. Emerg. Technol. Learn..

[5]  Winslow Burleson,et al.  Affect Recognition in Learning Scenarios: Matching Facial- and BCI-Based Values , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[6]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[7]  David Gibson,et al.  Data Challenges of Leveraging a Simulation to Assess Learning. , 2013 .

[8]  Mary E. Webb.  Beginning teacher education and collaborative formative e‐assessment , 2010 .

[9]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[10]  J. Adamson.  Handbook of Mixed Methods in Social and Behavioural Research. Tashakkori A, Teddlie C (eds). Thousand Oaks: Sage, 2003, pp.768, £77.00 ISBN: 0-7619-2073-0. , 2004 .

[11]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[12]  Ian Dunwell,et al.  Foundations of dynamic learning analytics: Using university student data to increase retention , 2015, Br. J. Educ. Technol..

[13]  Selen Turkay,et al.  Youth leadership development in virtual worlds: A case study , 2010 .

[14]  P. Robert-Jan Simons,et al.  The nature, reception, and use of online peer feedback in higher education , 2008, Comput. Educ..

[15]  Markus Hegland,et al.  The Apriori Algorithm – a Tutorial , 2005 .

[16]  C. Teddlie,et al.  SAGE Handbook of Mixed Methods in Social & Behavioral Research , 2010 .

[17]  Dirk Ifenthaler,et al.  Validation study of a method for assessing complex ill-structured problem solving by using causal representations , 2013, Educational Technology Research and Development.

[18]  Peggy A. Ertmer,et al.  Using Peer Feedback to Enhance the Quality of Student Online Postings: An Exploratory Study , 2007, J. Comput. Mediat. Commun..

[19]  Chris Dede,et al.  Assessment, Technology, and Change , 2010 .

[20]  Shameem Nyla.  NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION , 2004 .

[21]  Yoram M. Kalman,et al.  Pauses and Response Latencies: A Chronemic Analysis of Asynchronous CMC , 2006, J. Comput. Mediat. Commun..

[22]  Robert J. Mislevy,et al.  Evidence-Centered Design of Epistemic Games: Measurement Principles for Complex Learning Environments. , 2010 .

[23]  Amy Soller,et al.  Probabilities and predictions: modeling the development of scientific problem-solving skills. , 2005, Cell biology education.

[24]  Milind Tambe,et al.  Automated Assistants for Analyzing Team Behaviors , 2004, Autonomous Agents and Multi-Agent Systems.

[25]  James M. Boyle,et al.  A systematic literature review of empirical evidence on computer games and serious games , 2012, Comput. Educ..

[26]  Luisa Turrin Fernholz,et al.  Exploratory data analysis , 2018 .

[27]  Robert J. Mislevy,et al.  Evidence-Centered Design for Simulation-Based Assessment. CRESST Report 800. , 2011 .

[28]  Helmut Krcmar,et al.  Big Data , 2014, Wirtschaftsinf..

[29]  Athanasios V. Vasilakos,et al.  Big data: From beginning to future , 2016, Int. J. Inf. Manag..

[30]  Jody Clarke,et al.  Measuring inquiry: new methods, promises & challenges , 2010, ICLS.

[31]  David Gibson,et al.  simSchool: an online dynamic simulator for enhancing teacher preparation , 2011, Int. J. Learn. Technol..

[32]  Béatrice S. Hasler,et al.  Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds , 2013, Comput. Hum. Behav..

[33]  Michael D. Schmidt,et al.  Symbolic Regression of Implicit Equations , 2010 .

[34]  O. Sporns.  Networks of the Brain , 2010 .

[35]  George Siemens,et al.  Learning analytics: envisioning a research discipline and a domain of practice , 2012, LAK.

[36]  Thomas W. Morris.  Conversational Agents for Game-Like Virtual Environments , 2002 .

[37]  Sally Jordan,et al.  Assessment for learning: pushing the boundaries of computer-based assessment , 2009 .

[38]  Jody Clarke-Midura,et al.  Some Psychometric and Design Implications of Game Based Learning Analytics , 2015 .

[39]  R. Mislevy.  Evidence-centered design for simulation-based assessment. , 2011, Military medicine.

[40]  J. Michael Spector,et al.  E-Learning Systems, Environments and Approaches: Theory and Implementation , 2015 .

[41]  J. Hoffmann,et al.  Automatische Bewertung offener Antworten mittels Latenter Semantischer Analyse , 2007 .

[42]  Jody Clarke‐Midura,et al.  Technology Based Assessment for 21st Century Skills: Theoretical and Practical Implications from Modern Research , 2012 .

[43]  Naoto Kume,et al.  Asynchronous Teaching of Psychomotor Skills Through VR Annotations: Evaluation in Digital Rectal Examination , 2008, MMVR.

[44]  P. Kline.  The New Psychometrics: Science, Psychology and Measurement , 1998 .

[45]  Thomas A. Keenan.  Computers and education , 1964, CACM.

[46]  Rebecca Ferguson,et al.  Social Learning Analytics , 2012, J. Educ. Technol. Soc..

[47]  Jodi L. Davenport,et al.  21st Century Dynamic Assessment , 2012 .

[48]  Robert J. Mislevy,et al.  An evidence centered design for learning and assessment in the digital world , 2010 .

[49]  K. Perreault,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2011 .

[50]  Jiawei Han,et al.  Frequent pattern mining: current status and future directions , 2007, Data Mining and Knowledge Discovery.

[51]  Michael A. Rodriguez,et al.  Clickstream Data Yields High-Resolution Maps of Science , 2009, PloS one.