Development and Adoption of an Adaptive Learning System: Reflections and Lessons Learned

Adaptive learning systems (ALSs) aim to provide an efficient, effective and customised learning experience for students by dynamically adapting learning content to suit their individual abilities or preferences. Despite consistent evidence of their effectiveness and success in improving student learning over the past three decades, the actual impact and adoption of ALSs in education remain restricted to mostly research projects. In this paper, we provide a brief overview of reflections and lessons learned from developing and piloting an ALS in a course on relational databases. While our focus has been on adaptive learning, many of the presented lessons are also applicable to the development and adoption of educational tools and technologies in general. Our aim is to provide insight for other instructors, educational researchers and developers that are interested in adopting ALSs or are involved in the implementation of educational tools and technologies.

[1]  Vincent Aleven,et al.  A New Paradigm for Intelligent Tutoring Systems: Example-Tracing Tutors , 2009, Int. J. Artif. Intell. Educ..

[2]  John R. Anderson,et al.  Cognitive Tutors: Lessons Learned , 1995 .

[3]  David Boud,et al.  Developing evaluative judgement: enabling students to make decisions about the quality of work , 2018 .

[4]  Mollie Dollinger,et al.  Co-creation strategies for learning analytics , 2018, LAK.

[5]  Neil T. Heffernan,et al.  The ASSISTments Ecosystem: Building a Platform that Brings Scientists and Teachers Together for Minimally Invasive Research on Human Learning and Teaching , 2014, International Journal of Artificial Intelligence in Education.

[6]  Aslina Saad,et al.  Gamification Elements and Their Impacts on Teaching and Learning – A Review , 2018 .

[7]  Simon Burns,et al.  Doing it for themselves: students creating a high quality peer-learning environment , 2015 .

[8]  Tom Routen,et al.  Intelligent Tutoring Systems , 1996, Lecture Notes in Computer Science.

[9]  Burak Yilmaz,et al.  Effects of Adaptive Learning Technologies on Math Achievement: A Quantitative Study of ALEKS Math Software , 2017 .

[10]  Mitchell J. Nathan,et al.  Expert Blind Spot : When Content Knowledge Eclipses Pedagogical Content Knowledge , 2001 .

[11]  Fernando González-Ladrón-de-Guevara,et al.  Towards an integrated crowdsourcing definition , 2012, J. Inf. Sci..

[12]  Dragan Gasevic,et al.  A Multivariate ELO-based Learner Model for Adaptive Educational Systems , 2019, EDM.

[13]  Simon Bates,et al.  Assessing the quality of a student-generated question repository , 2013, 1308.2202.

[14]  Olusola O. Adesope,et al.  Intelligent tutoring systems and learning outcomes: A meta-analysis , 2014 .

[15]  Hendrik Drachsler,et al.  Privacy and analytics: it's a DELICATE issue a checklist for trusted learning analytics , 2016, LAK.

[16]  Hassan Khosravi,et al.  RiPLE: Recommendation in Peer-Learning Environments Based on Knowledge Gaps and Interests , 2017, EDM.

[17]  Ryan S. Baker,et al.  Studying Adaptive Learning Efficacy using Propensity Score Matching , 2018 .

[18]  Judy Kay,et al.  Open learner models and learning analytics dashboards: a systematic review , 2018, LAK.

[19]  Neil T. Heffernan,et al.  The Future of Adaptive Learning: Does the Crowd Hold the Key? , 2016, International Journal of Artificial Intelligence in Education.

[20]  James H. McMillan,et al.  Research in Education: Evidence Based Inquiry , 2005 .

[21]  Neil T. Heffernan,et al.  AXIS: Generating Explanations at Scale with Learnersourcing and Machine Learning , 2016, L@S.

[22]  James H. McMillan,et al.  Research in Education: Evidence-Based Inquiry, 7th Edition. MyEducationLab Series. , 2010 .

[23]  Carole Torgerson,et al.  Randomised controlled trials (RCTs) in education research –methodological debates, questions, challenges , 2018, Educational Research.

[24]  Rebecca Ferguson,et al.  Guest Editorial: Ethics and Privacy in Learning Analytics , 2016, J. Learn. Anal..

[25]  Gail M Sullivan,et al.  Getting off the "gold standard": randomized controlled trials and education research. , 2011, Journal of graduate medical education.

[26]  K. VanLehn.  The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems , 2011 .

[27]  Catherine Mulryan-Kyne,et al.  Teaching large classes at college and university level: challenges and opportunities , 2010 .

[28]  Vincent Aleven,et al.  Instruction Based on Adaptive Learning Technologies , 2016 .

[29]  D. Rubin,et al.  The central role of the propensity score in observational studies for causal effects , 1983 .

[30]  Joseph Jay Williams,et al.  RiPPLE: A Crowdsourced Adaptive Platform for Recommendation of Learning Activities , 2019, J. Learn. Anal..

[31]  Eileen Wood,et al.  Encouraging Mindful Use of Prior Knowledge: Attempting to Construct Explanatory Answers Facilitates Learning , 1992 .

[32]  Kelly E. Matthews,et al.  Five Propositions for Genuine Students as Partners Practice , 2017 .

[33]  J. Bransford,et al.  Preparing Teachers for a Changing World: What Teachers Should Learn and Be Able to Do. , 2005 .

[34]  Paul Denny,et al.  Empirical Support for a Causal Relationship Between Gamification and Learning Outcomes , 2018, CHI.

[35]  Erik Duval,et al.  Visualizing recommendations to support exploration, transparency and controllability , 2013, IUI '13.

[36]  Elizabeth A. Linnenbrink,et al.  Motivation as an Enabler for Academic Success , 2002 .

[37]  Linda Corrin,et al.  The Ethics of Learning Analytics in Australian Higher Education: Discussion Paper , 2019 .

[38]  Hassan Khosravi,et al.  Topic Dependency Models: Graph-Based Visual Analytics for Communicating Assessment Data , 2018, J. Learn. Anal..

[39]  Larry Cuban.  Rethinking education in the age of technology: The digital revolution and schooling in America , 2010 .

[40]  Maren Scheffel,et al.  License to evaluate: preparing learning analytics dashboards for educational practice , 2018, LAK.

[41]  Judy Kay,et al.  Open Learner Models , 2010, Advances in Intelligent Tutoring Systems.

[42]  John Hamer,et al.  PeerWise: students sharing their multiple choice questions , 2008, ICER '08.

[43]  Grant T. Harris,et al.  Comparing Effect Sizes in Follow-Up Studies: ROC Area, Cohen's d, and r , 2005, Law and human behavior.

[44]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[45]  Paul Denny,et al.  Formative student-authored question bank: perceptions, question quality and association with summative performance , 2017, Postgraduate Medical Journal.

[46]  John A. Self,et al.  System Intelligence in Constructivist Learning , 2000 .

[47]  Ok-Choon Park,et al.  Adaptive Instructional Systems , 2007 .

[48]  Rishi Desai,et al.  Crowdsourcing for assessment items to support adaptive learning , 2018, Medical teacher.

[49]  Kurt VanLehn,et al.  The Behavior of Tutoring Systems , 2006, Int. J. Artif. Intell. Educ..

[50]  Boualem Benatallah,et al.  A Review on Crowdsourcing for Education: State of the Art of Literature and Practice , 2018, PACIS.

[51]  Bieke Zaman,et al.  Need-supporting gamification in education: An assessment of motivational effects over time , 2018, Comput. Educ..

[52]  Seiji Isotani,et al.  A systematic mapping on gamification applied to education , 2014, SAC.

[53]  Zhihong Xu,et al.  The effectiveness of intelligent tutoring systems on K-12 students' reading comprehension: A meta-analysis , 2019, Br. J. Educ. Technol..

[54]  Vladimir Zadorozhny,et al.  Adaptive Social Learning Based on Crowdsourcing , 2017, IEEE Transactions on Learning Technologies.