Inferring Learning from Big Data: The Importance of a Transdisciplinary and Multidimensional Approach

The use of big data in higher education has evolved rapidly with a focus on the practical application of new tools and methods for supporting learning. In this paper, we depart from the core emphasis on application and delve into a mostly neglected aspect of the big data conversation in higher education. Drawing on developments in cognate disciplines, we analyse the inherent difficulties in inferring the complex phenomenon that is learning from big datasets. This forms the basis of a discussion about the possibilities for systematic collaboration across different paradigms and disciplinary backgrounds in interpreting big data for enhancing learning. The aim of this paper is to provide the foundation for a research agenda, where differing conceptualisations of learning become a strength in interpreting patterns in big datasets, rather than a point of contention.

[1]  Amanda Datnow,et al.  Teacher capacity for and beliefs about data-driven decision making: A literature review of international research , 2015, Journal of Educational Change.

[2]  Thomas C. Reeves,et al.  Design research from a technology perspective , 2006 .

[3]  D. Rowntree.  Assessing Students: How Shall We Know Them? , 1977 .

[4]  Gregory M. Donoghue,et al.  A Bridge Too Far – Revisited: Reframing Bruer’s Neuroeducation Argument for Modern Science of Learning Practitioners , 2016, Front. Psychol..

[5]  Jason M. Lodge,et al.  A FRAMEWORK FOR ORGANIZING AND TRANSLATING SCIENCE OF LEARNING RESEARCH , 2017 .

[6]  Annie Y. S. Lau,et al.  The influence of social networking sites on health behavior change: a systematic review and meta-analysis , 2015, J. Am. Medical Informatics Assoc..

[7]  George Siemens,et al.  Learning Analytics , 2013 .

[8]  Paul Prinsloo,et al.  Student Vulnerability, Agency and Learning Analytics: An Exploration , 2016, J. Learn. Anal..

[9]  R. Bjork,et al.  Learning Versus Performance , 2013, Perspectives on psychological science : a journal of the Association for Psychological Science.

[10]  T. Yarkoni.  Psychoinformatics: New Horizons at the Interface of the Psychological and Computing Sciences , 2012 .

[11]  Danny Y. T. Liu,et al.  Learning analytics - are we at risk of missing the point? , 2015 .

[12]  Jason M. Lodge,et al.  Pigeon pecks and mouse clicks: Putting the learning back into learning analytics , 2012 .

[13]  Julie A. Marsh,et al.  Contributing conditions: A qualitative comparative analysis of teachers’ instructional responses to data , 2016 .

[14]  Peter Reimann,et al.  Connecting learning analytics with learning research: the role of design-based research , 2016 .

[15]  Manu Kapur,et al.  Conceptualizing Debates in Learning and Educational Research: Toward a Complex Systems Conceptual Framework of Learning , 2016 .

[16]  Tim Rogers,et al.  Critical realism and learning analytics research: epistemological implications of an ontological foundation , 2015, LAK.

[17]  John H. Holland,et al.  Induction: Processes of Inference, Learning, and Discovery , 1987, IEEE Expert.

[18]  Manfred Hauben,et al.  Illusions of objectivity and a recommendation for reporting data mining results , 2007, European Journal of Clinical Pharmacology.

[19]  H. Simon,et al.  Situated Learning and Education1 , 1996 .

[20]  Troubling the edTPA: Illusions of Objectivity and Rigor , 2016 .

[21]  G. Kennedy,et al.  Stepping beyond the paradigm wars: pluralist methods for research in learning technology , 2011 .

[22]  Jason M. Lodge,et al.  Keep Calm and Credential on: Linking Learning, Life and Work Practices in a Complex World , 2016 .

[23]  Leah P. Macfadyen,et al.  Embracing Big Data in Complex Educational Systems: The Learning Analytics Imperative and the Policy Challenge. , 2014 .

[24]  Russell E. Glasgow,et al.  Evaluating the Relevance, Generalization, and Applicability of Research , 2006, Evaluation & the health professions.

[25]  R. Barnett.  Learning for an unknown future , 2004 .

[26]  Rebecca Ferguson,et al.  Guest Editorial: Ethics and Privacy in Learning Analytics , 2016, J. Learn. Anal..

[27]  A. Wise,et al.  Why Theory Matters More than Ever in the Age of Big Data , 2015, J. Learn. Anal..

[28]  Geoffrey L. Cohen,et al.  Changing Environments by Changing Individuals , 2016, Psychological science.

[29]  Jo Rycroft-Malone,et al.  Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges , 2008, Implementation science : IS.

[30]  Richard Arum,et al.  Academically Adrift: Limited Learning on College Campuses , 2011 .

[31]  R. Ackoff.  From Data to Wisdom , 2014 .

[32]  George Siemens,et al.  Learning analytics and educational data mining: towards communication and collaboration , 2012, LAK.

[33]  Jason M. Lodge,et al.  From the Laboratory to the Classroom : Translating Science of Learning for Teachers , 2017 .

[34]  Thad Dunning,et al.  Improving Causal Inference , 2008 .

[35]  Ryan Shaun Joazeiro de Baker,et al.  Stupid Tutoring Systems, Intelligent Humans , 2016, International Journal of Artificial Intelligence in Education.

[36]  A Kitson,et al.  Enabling the implementation of evidence based practice: a conceptual framework. , 1998, Quality in health care : QHC.

[37]  S. Maxwell,et al.  Is psychology suffering from a replication crisis? What does "failure to replicate" really mean? , 2015, The American psychologist.

[38]  B. Nicolescu.  Manifesto of Transdisciplinarity , 2002 .

[39]  Zachary A. Pardos,et al.  The sum is greater than the parts: ensembling models of student knowledge in educational software , 2012, SKDD.

[40]  George Siemens,et al.  Guest Editorial - Learning and Knowledge Analytics , 2012, J. Educ. Technol. Soc..

[41]  Robert J. Mislevy,et al.  The role of probability-based inference in an intelligent tutoring system , 2005, User Modeling and User-Adapted Interaction.

[42]  Karen Littleton,et al.  Epistemology, Assessment, Pedagogy: Where Learning Meets Analytics in the Middle Space , 2014, J. Learn. Anal..

[43]  D. Krathwohl.  A Revision of Bloom's Taxonomy: An Overview , 2002 .

[44]  George Siemens,et al.  Let’s not forget: Learning analytics are about learning , 2015 .

[45]  J. de Houwer,et al.  What is learning? On the nature and merits of a functional definition of learning , 2013, Psychonomic Bulletin & Review.

[46]  David C. Berliner,et al.  Collateral Damage: How High-Stakes Testing Corrupts America's Schools , 2008 .

[47]  Simon Buckingham Shum,et al.  Learning Analytics for 21st Century Competencies , 2016, J. Learn. Anal..

[48]  Sandra Milligan,et al.  Crowd-sourced learning in MOOCs: learning analytics meets measurement theory , 2015, LAK.

[49]  A. Newell.  Unified Theories of Cognition , 1990 .

[50]  Susan M. Land,et al.  Theoretical Foundations of Learning Environments. , 1999 .

[51]  J. Horvath,et al.  The hard problem of ‘educational neuroscience’ , 2017, Trends in Neuroscience and Education.

[52]  Jacob Cohen.  The earth is round (p < .05) , 1994 .

[53]  Paul A Estabrooks,et al.  Assessing the Internal and External Validity of Mobile Health Physical Activity Promotion Interventions: A Systematic Literature Review Using the RE-AIM Framework , 2013, Journal of medical Internet research.