Meta-analysis of the impact of Augmented Reality on students’ learning gains

Abstract Existing literature reflects the multiple benefits of the integration of Augmented Reality (AR) technologies in educational settings. Many studies have been conducted to establish the tendencies, affordances, and challenges of this technology for education. However, most of these studies are qualitative studies that do not measure the extent of the impact of this technology on education. This study conducted a meta-analysis of 64 quantitative research papers ( N  = 4705) published between 2010 and 2018 in major journals. The main purpose of the study was to analyze the impact of AR on students’ learning gains. Furthermore, the study analyzed the influence of moderating variables such as control treatment, learning environment, learner type, and domain subject on the learning gains. The results identified that AR has a medium effect on the learning gains of students ( d = . 68 , p . 001 ). Additionally, the study discusses the effect of AR on the moderating variables and establishes some possible routes of investigation for future work.

[1]  Yi-Hsuan Wang,et al.  Exploring the effectiveness of integrating augmented reality-based materials to support writing activities , 2017, Comput. Educ..

[2]  Manuel Contero,et al.  Using an Augmented Reality Enhanced Tabletop System to Promote Learning of Mathematics: A Case Study with Students with Special Educational Needs. , 2017 .

[3]  Jorge Martín Gutiérrez,et al.  Applying augmented reality in engineering education to improve academic performance & student motivation , 2014 .

[4]  Mariano Alcañiz Raya,et al.  Design and validation of an augmented book for spatial abilities development in engineering students , 2010, Comput. Graph..

[5]  R. Moreno.  Decreasing Cognitive Load for Novice Students: Effects of Explanatory versus Corrective Feedback in Discovery-Based Multimedia , 2004 .

[6]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[7]  Philip C. Abrami,et al.  What Forty Years of Research Says About the Impact of Technology on Learning , 2011 .

[8]  Rabia Yilmaz,et al.  Using augmented reality technology in storytelling activities: examining elementary students’ narrative skill and creativity , 2016, Virtual Reality.

[9]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[10]  Yao-Ting Sung,et al.  Apply an Augmented Reality in a Mobile Guidance to Increase Sense of Place for Heritage Places , 2015, J. Educ. Technol. Soc..

[11]  Tsung-Yu Liu,et al.  Using ubiquitous games in an English listening and speaking course: Impact on learning outcomes and motivation , 2010, Comput. Educ..

[12]  S. Thompson,et al.  Quantifying heterogeneity in a meta‐analysis , 2002, Statistics in medicine.

[13]  Hannah R Rothstein,et al.  A basic introduction to fixed‐effect and random‐effects models for meta‐analysis , 2010, Research synthesis methods.

[14]  Silvia Baldiris,et al.  ARtour: Augmented Reality-Based Game to Promote Agritourism , 2018, AVR.

[15]  S.-W. Huang,et al.  Applying augmented reality to enhance learning: a study of different teaching materials , 2017, J. Comput. Assist. Learn..

[16]  Silvia Baldiris,et al.  Augmented Reality Game-Based Learning: Enriching Students’ Experience During Reading Comprehension Activities , 2017 .

[17]  I. Widiaty,et al.  MOBILE-BASED AUGMENTED REALITY FOR LEARNING 3-DIMENSIONAL SPATIAL BATIK-BASED OBJECTS , 2017 .

[18]  Ken Kelley,et al.  On effect size. , 2012, Psychological methods.

[19]  Anne-Wil Harzing,et al.  Google Scholar, Scopus and the Web of Science: a longitudinal and cross-disciplinary comparison , 2015, Scientometrics.

[20]  Chien-Hsu Chen,et al.  Augmented reality-based video-modeling storybook of nonverbal facial cues for children with autism spectrum disorder to improve their perceptions and judgments of facial expressions and emotions , 2016, Comput. Hum. Behav..

[21]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[22]  Kuei-Fang Hsiao.  Using augmented reality for students health - case of combining educational learning with standard fitness , 2011, Multimedia Tools and Applications.

[23]  Y.-H. Wang.  Using augmented reality to support a software editing course for college students , 2017, J. Comput. Assist. Learn..

[24]  Deryn Watson,et al.  Pedagogy before Technology: Re-thinking the Relationship between ICT and Teaching , 2001, Education and Information Technologies.

[25]  Jessica Gurevitch,et al.  Meta-analysis and the science of research synthesis , 2018, Nature.

[26]  Mark Billinghurst,et al.  A Survey of Augmented Reality , 2015, Found. Trends Hum. Comput. Interact..

[27]  Ting-Kwei Wang,et al.  Does Augmented Reality Effectively Foster Visual Learning Process in Construction? An Eye-Tracking Study in Steel Installation , 2018, Advances in Civil Engineering.

[28]  Sukhyun Lim,et al.  An augmented reality-based authoring tool for E-learning applications , 2011, Multimedia Tools and Applications.

[29]  Charles T. Jahren,et al.  A systematic review of research on the flipped learning method in engineering education , 2018, Br. J. Educ. Technol..

[30]  Betsy Jane Becker,et al.  Synthesizing standardized mean‐change measures , 1988 .

[31]  M. Carmen Juan,et al.  A Mobile Augmented Reality System for the Learning of Dental Morphology , 2016 .

[32]  M. Özdemir,et al.  The Effect of Augmented Reality Applications in the Learning Process: A Meta-Analysis Study , 2018 .

[33]  Sylvie Daniel,et al.  Observation and analysis of a classroom teaching and learning practice based on augmented reality and serious games on mobile platforms , 2015, Int. J. Serious Games.

[34]  D. Lewis,et al.  Quantitative methods in psychology , 1950 .

[35]  Zeynel Abidin Yılmaz,et al.  Artırılmış Gerçeklik Uygulamalarının Eğitimle Bütünleştirilmesinin Meta-Analitik ve Tematik Karşılaştırmalı Analizi , 2016 .

[36]  Wernhuar Tarng,et al.  Development of a Lunar-Phase Observation System Based on Augmented Reality and Mobile Learning Technologies , 2016, Mob. Inf. Syst..

[37]  Iulian Radu,et al.  Augmented reality in education: a meta-review and cross-media analysis , 2014, Personal and Ubiquitous Computing.

[38]  Carlos Delgado Kloos,et al.  Support for Augmented Reality Simulation Systems: The Effects of Scaffolding on Learning Outcomes and Behavior Patterns , 2016, IEEE Transactions on Learning Technologies.

[39]  Carlos Delgado Kloos,et al.  Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness , 2014, Comput. Educ..

[40]  Chris Dede,et al.  Affordances and Limitations of Immersive Participatory Augmented Reality Simulations for Teaching and Learning , 2009 .

[41]  Fengfeng Ke,et al.  Mobile augmented-reality artifact creation as a component of mobile computer-supported collaborative learning , 2015, Internet High. Educ..

[42]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[43]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[44]  Antonia Mireles Medina,et al.  Planning and Allocation of Digital Learning Objects with Augmented Reality to Higher Education Students According to the VARK Model , 2018, Int. J. Interact. Multim. Artif. Intell..

[45]  Muhammad Munwar Iqbal,et al.  An E-Assessment Framework for Blended Learning with Augmented Reality to Enhance the Student Learning , 2017 .

[46]  Robert Levine,et al.  Interactive Anatomy-Augmented Virtual Simulation Training. , 2018, Clinical simulation in nursing.

[47]  Francisco J. García-Peñalvo,et al.  Augmented reality and pedestrian navigation through its implementation in m-learning and e-learning: Evaluation of an educational program in Chile , 2017, Comput. Educ..

[48]  S. Sawilowsky.  New Effect Size Rules of Thumb , 2009 .

[49]  R. Clark.  Media will never influence learning , 1994 .

[50]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[51]  Hsin-Kai Wu,et al.  Designing Applications for Physics Learning: Facilitating High School Students' Conceptual Understanding by Using Tablet PCS , 2015 .

[52]  Chih-Kai Chang,et al.  Assessing the effectiveness of learning solid geometry by using an augmented reality-assisted learning system , 2015, Interact. Learn. Environ..

[53]  Andrés-Marcelo Calle-Bustos,et al.  An augmented reality game to support therapeutic education for children with diabetes , 2017, PloS one.

[54]  Wernhuar Tarng,et al.  A Sun Path Observation System Based on Augment Reality and Mobile Learning , 2018, Mob. Inf. Syst..

[55]  Mehdi Mekni,et al.  Augmented Reality : Applications , Challenges and Future Trends , 2014 .

[56]  Shih-Chung Kang,et al.  Use of Tangible and Augmented Reality Models in Engineering Graphics Courses , 2011 .

[57]  Hsiao-Ching She,et al.  Learning through playing Virtual Age: Exploring the interactions among student concept learning, gaming performance, in-game behaviors, and the use of in-game characters , 2015, Comput. Educ..

[58]  Ting-Chia Hsu,et al.  Learning English with Augmented Reality: Do learning styles matter? , 2017, Comput. Educ..

[59]  Michelle K. Smith,et al.  Active learning increases student performance in science, engineering, and mathematics , 2014, Proceedings of the National Academy of Sciences.

[60]  Carlos Delgado Kloos,et al.  Impact of an augmented reality system on students' motivation for a visual art course , 2013, Comput. Educ..

[61]  Feng-Kuang Chiang,et al.  Using the Augmented Reality 3D Technique for a Convex Imaging Experiment in a Physics Course , 2013 .

[62]  Noel Enyedy,et al.  Learning physics through play in an augmented reality environment , 2012, Int. J. Comput. Support. Collab. Learn..

[63]  Xu Wang,et al.  A case study of Augmented Reality simulation system application in a chemistry course , 2014, Comput. Hum. Behav..

[64]  Cheng-ping Chen,et al.  Employing Augmented-Reality-Embedded Instruction to Disperse the Imparities of Individual Differences in Earth Science Learning , 2015 .

[65]  S. Morris.  Estimating Effect Sizes From Pretest-Posttest-Control Group Designs , 2008 .

[66]  Bilal Atasoy,et al.  Artırılmış Gerçeklik Uygulamalarının İlköğretim Öğrencilerinin Uzamsal Yeteneklerine ve Akademik Başarılarına Etkisi , 2017 .

[67]  Dirk Basten,et al.  Benefits of Augmented Reality in Educational Environments - A Systematic Literature Review , 2015, Wirtschaftsinformatik.

[68]  Donald Hedeker,et al.  Estimation of Effect Size From a Series of Experiments Involving Paired Comparisons , 1993 .

[69]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[70]  Gwo-Jen Hwang,et al.  Impacts of an augmented reality-based flipped learning guiding approach on students' scientific project performance and perceptions , 2018, Comput. Educ..

[71]  Noel A. Card.  Applied Meta-Analysis for Social Science Research , 2011 .

[72]  Paul L. Sauer,et al.  Is Augmented Reality Technology an Effective Tool for E-commerce? An Interactivity and Vividness Perspective , 2017 .

[73]  Hsin-Yi Chang,et al.  Integrating a mobile augmented reality activity to contextualize student learning of a socioscientific issue , 2013, Br. J. Educ. Technol..

[74]  Jia Zhang,et al.  The development and evaluation of an augmented reality-based armillary sphere for astronomical observation instruction , 2014, Comput. Educ..

[75]  Gwo-Jen Hwang,et al.  Impacts of integrating the repertory grid into an augmented reality-based learning design on students’ learning achievements, cognitive load and degree of satisfaction , 2018, Interact. Learn. Environ..

[76]  Murat Akçayır,et al.  Advantages and challenges associated with augmented reality for education: A systematic review of the literature , 2017 .

[77]  Zahira Merchant,et al.  Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis , 2014, Comput. Educ..

[78]  Hakan Tekedere,et al.  Examining the Effectiveness of Augmented Reality Applications in Education: A Meta-Analysis. , 2016 .

[79]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[80]  Mehmet Akif Ocak,et al.  Augmented reality in science laboratories: The effects of augmented reality on university students' laboratory skills and attitudes toward science laboratories , 2016, Comput. Hum. Behav..

[81]  Sherry Y. Chen,et al.  The Influences of the 2D Image-Based Augmented Reality and Virtual Reality on Student Learning , 2017, J. Educ. Technol. Soc..

[82]  Richard E. Clark,et al.  Why Minimal Guidance During Instruction Does Not Work: An Analysis of the Failure of Constructivist, Discovery, Problem-Based, Experiential, and Inquiry-Based Teaching , 2006 .

[83]  R. Kozma.  Will media influence learning? Reframing the debate , 1994 .

[84]  Roland Klemke,et al.  Attuning a mobile simulation game for school children using a design-based research approach , 2015, Comput. Educ..

[85]  S Duval,et al.  Trim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis , 2000, Biometrics.

[86]  Damla Karagozlu,et al.  Determination of the impact of augmented reality application on the success and problem-solving skills of students , 2018 .

[87]  Tania B. Huedo-Medina,et al.  Assessing heterogeneity in meta-analysis: Q statistic or I2 index? , 2006, Psychological methods.

[88]  Gwo-Jen Hwang,et al.  Influences of an inquiry-based ubiquitous gaming design on students' learning achievements, motivation, behavioral patterns, and tendency towards critical thinking and problem solving , 2017, Br. J. Educ. Technol..

[89]  Chien-Hsu Chen,et al.  An Augmented-Reality-Based Concept Map to Support Mobile Learning for Science , 2016, The Asia-Pacific Education Researcher.

[90]  R. Mayer.  Should there be a three-strikes rule against pure discovery learning? The case for guided methods of instruction. , 2004, The American psychologist.

[91]  June Lee,et al.  Effectiveness of Computer-Based Instructional Simulation: A Meta Analysis. , 1999 .

[92]  L. Hedges.  Estimation of effect size from a series of independent experiments. , 1982 .

[93]  Silke L. Schneider.  The International Standard Classification of Education 2011 , 2013 .

[94]  Manuel Castro,et al.  New technology trends in education: Seven years of forecasts and convergence , 2011, Comput. Educ..

[95]  Traci Sitzmann.  A META-ANALYTIC EXAMINATION OF THE INSTRUCTIONAL EFFECTIVENESS OF COMPUTER-BASED SIMULATION GAMES , 2011 .

[96]  Henry Been-Lirn Duh,et al.  An investigation of learners' collaborative knowledge construction performances and behavior patterns in an augmented reality simulation system , 2013, Comput. Educ..

[97]  Mary Ratcliffe,et al.  The science and values that young people draw upon to make decisions about biological conservation issues , 2002 .

[98]  Ernesto Redondo Domínguez,et al.  Assessment of augmented visualization methods in multimedia engineering education , 2015 .

[99]  eynel Abidin Yılmaz,et al.  A Meta-Analytic and Thematic Comparative Analysis of the Integration of Augmented Reality Applications into Education , 2016 .

[100]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[101]  Su-ju Lu,et al.  Integrating augmented reality technology to enhance children’s learning in marine education , 2015 .

[102]  H. Pashler,et al.  Distributed practice in verbal recall tasks: A review and quantitative synthesis. , 2006, Psychological bulletin.

[103]  Wojciech Cellary,et al.  Evaluation of learners' attitude toward learning in ARIES augmented reality environments , 2013, Comput. Educ..

[104]  Jyh-Chong Liang,et al.  Current status, opportunities and challenges of augmented reality in education , 2013, Comput. Educ..

[105]  R. DeShon,et al.  Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. , 2002 .

[106]  Lorenzo Moreno Ruiz,et al.  Applying a constructivist and collaborative methodological approach in engineering education , 2007, Comput. Educ..

[107]  Chris Dede,et al.  EcoMOBILE: Integrating augmented reality and probeware with environmental education field trips , 2013, Comput. Educ..

[108]  W. Dunlap,et al.  Meta-Analysis of Experiments With Matched Groups or Repeated Measures Designs , 1996 .

[109]  Jie Chao,et al.  The effects of augmented virtual science laboratories on middle school students' understanding of gas properties , 2015, Comput. Educ..

[110]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[111]  Sumedha Chauhan,et al.  A meta-analysis of the impact of technology on learning effectiveness of elementary students , 2017, Comput. Educ..

[112]  E. Ushioda,et al.  Beyond global English : motivation to learn languages in a multicultural world : Introduction to the Special Issue , 2017 .

[113]  Oliver Müller,et al.  Augmented reality in informal learning environments: A field experiment in a mathematics exhibition , 2014, Comput. Educ..

[114]  Robert M. Bernard,et al.  How Does Distance Education Compare With Classroom Instruction? A Meta-Analysis of the Empirical Literature , 2004 .

[115]  Zeynep Turan,et al.  The impact of mobile augmented reality in geography education: achievements, cognitive loads and views of university students , 2018 .

[116]  Rebecca Ferguson,et al.  Augmented Reality and Mobile Learning: the State of the Art , 2013, mLearn.

[117]  Sevda Küçük,et al.  Learning anatomy via mobile augmented reality: Effects on achievement and cognitive load , 2016, Anatomical sciences education.

[118]  J. Ferrer-Torregrosa,et al.  ARBOOK: Development and Assessment of a Tool Based on Augmented Reality for Anatomy , 2015 .

[119]  Chih-Ming Chen,et al.  Interactive augmented reality system for enhancing library instruction in elementary schools , 2013, Comput. Educ..

[120]  Yao-Ting Sung,et al.  Development and behavioral pattern analysis of a mobile guide system with augmented reality for painting appreciation instruction in an art museum , 2014, Comput. Educ..

[121]  Jorge Bacca,et al.  Augmented Reality Trends in Education: A Systematic Review of Research and Applications , 2014, J. Educ. Technol. Soc..

[122]  Kevin D. Carlson,et al.  Impact of Experimental Design on Effect Size: Findings From the Research Literature on Training , 1999 .

[123]  Yu-Hui Ching,et al.  Trends of educational technology research: more than a decade of international research in six SSCI-indexed refereed journals , 2013, Educational Technology Research and Development.

[124]  John Schacter,et al.  Does Computer Technology Improve Student Learning and Achievement? How, When, and under What Conditions? , 1999 .

[125]  Nian-Shing Chen,et al.  Learning while exercising for science education in augmented reality among adolescents , 2012, Interact. Learn. Environ..

[126]  R. Rosenthal.  Meta-analytic procedures for social research , 1984 .

[127]  Ammar H. Safar,et al.  The Effectiveness of Using Augmented Reality Apps in Teaching the English Alphabet to Kindergarten Children: A Case Study in the State of Kuwait. , 2017 .

[128]  Ezgi Tosik Gün,et al.  The Effects of Augmented Reality on Elementary School Students ’ Spatial Ability and Academic Achievement , 2017 .

[129]  Recep Cakir,et al.  Exploring the Effect of Materials Designed with Augmented Reality on Language Learners' Vocabulary Learning. , 2015 .

[130]  Silvia Baldiris,et al.  Systematic review and meta-analysis of augmented reality in educational settings , 2019, Virtual Reality.

[131]  Hossein Nouri,et al.  Effect Size for ANOVA Designs , 1999 .

[132]  Feng-Kuang Chiang,et al.  Applications of augmented reality-based natural interactive learning in magnetic field instruction , 2017, Interact. Learn. Environ..

[133]  Wernhuar Tarng,et al.  Development of a virtual butterfly ecological system based on augmented reality and mobile learning technologies , 2013, 2013 9th International Wireless Communications and Mobile Computing Conference (IWCMC).

[134]  Wen-Hung Chao,et al.  Using Augmented Reality to Enhance and Engage Students in Learning Mathematics , 2018, Advances in Social Sciences Research Journal.

[135]  Journals Invention,et al.  The Pedagogical Potential of Augmented Reality Apps , 2015 .

[136]  Monica W. Tracey,et al.  Does media affect learning: where are we now? , 2005 .

[137]  Hirokazu Kato,et al.  Augmented Reality Learning Experiences: Survey of Prototype Design and Evaluation , 2014, IEEE Transactions on Learning Technologies.

[138]  Chia-Chen Chen,et al.  Animating eco-education: To see, feel, and discover in an augmented reality-based experiential learning environment , 2016, Comput. Educ..