Augmented reality as multimedia: the case for situated vocabulary learning

Augmented reality (AR) has the potential to create compelling learning experiences. However, there are few research works exploring the design and evaluation of AR for educational settings. In our research, we treat AR as a type of multimedia that is situated in authentic environments and apply multimedia learning theory as a framework for developing our educational applications. We share our experiences in developing a handheld AR system and one specific use case, namely, situated vocabulary learning. Results of our evaluations show that we are able to create AR applications with good system usability. More importantly, our preliminary evaluations show that AR may lead to better retention of words and improve student attention and satisfaction.

[1]  Khai N. Truong,et al.  Evaluating the implicit acquisition of second language vocabulary using a live wallpaper , 2012, CHI.

[2]  Mark Billinghurst,et al.  Augmented Reality in the Classroom , 2012, Computer.

[3]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[4]  Chih-Ming Chen,et al.  Context-Aware Ubiquitous English Learning in a Campus Environment , 2007, Seventh IEEE International Conference on Advanced Learning Technologies (ICALT 2007).

[5]  Heidi A. Diefes-Dux,et al.  A preliminary validation of Attention, Relevance, Confidence and Satisfaction model-based Instructional Material Motivational Survey in a computer-based tutorial setting , 2006, Br. J. Educ. Technol..

[6]  S. Matsutomo,et al.  Real-Time Visualization System of Magnetic Field Utilizing Augmented Reality Technology for Education , 2012, IEEE Transactions on Magnetics.

[7]  Fang-Chuan Ou Yang,et al.  Using Personalized VLS on Mobile English Vocabulary Learning , 2012, 2012 IEEE Seventh International Conference on Wireless, Mobile and Ubiquitous Technology in Education.

[8]  J. Keller.  Development and use of the ARCS model of instructional design , 1987 .

[9]  Jing Zhao,et al.  MicroMandarin: mobile language learning in context , 2011, CHI.

[10]  Hannes Kaufmann.  Construct3D: an augmented reality application for mathematics and geometry education , 2002, MULTIMEDIA '02.

[11]  Christian Sandor,et al.  Authoring augmented reality as situated multimedia , 2014 .

[12]  Anil Kumar Singh,et al.  Games for Academic Vocabulary Learning through a Virtual Environment , 2011, 2011 International Conference on Asian Language Processing.

[13]  Ziguang Zheng,et al.  An Approach to Context-Aware Mobile Chinese Language Learning for Foreign Students , 2009, 2009 Eighth International Conference on Mobile Business.

[14]  Carrie Demmans Epp.  Mobile Adaptive Communication Support for Vocabulary Acquisition , 2013, AIED.

[15]  J. Edward Swan,et al.  Usability Engineering for Augmented Reality: Employing User-Based Studies to Inform Design , 2008, IEEE Transactions on Visualization and Computer Graphics.

[16]  Wernhuar Tarng,et al.  A Study of Campus Butterfly Ecology Learning System Based on Augmented Reality and Mobile Learning , 2012, 2012 IEEE Seventh International Conference on Wireless, Mobile and Ubiquitous Technology in Education.

[17]  Dieter Schmalstieg,et al.  Construct3D: A Virtual Reality Application for Mathematics and Geometry Education , 2000, Education and Information Technologies.

[18]  Blair MacIntyre,et al.  DART: a toolkit for rapid design exploration of augmented reality experiences , 2005, ACM Trans. Graph..

[19]  Jeff Sauro,et al.  The Factor Structure of the System Usability Scale , 2009, HCI.

[20]  Shelley Shwu-Ching Young,et al.  The Game-Based Constructive Learning Environment to Increase English Vocabulary Acquisition: Implementing a Wireless Crossword Fan-Tan Game (WiCFG) as an Example , 2008, Fifth IEEE International Conference on Wireless, Mobile, and Ubiquitous Technology in Education (wmute 2008).

[21]  Daniel D. Suthers,et al.  PhotoStudy: vocabulary learning and collaboration on fixed & mobile devices , 2005, IEEE International Workshop on Wireless and Mobile Technologies in Education (WMTE'05).

[22]  Goshiro Yamamoto,et al.  Augmented Reality X-Ray Interaction in K-12 Education: Theory, Student Perception and Teacher Evaluation , 2013, 2013 IEEE 13th International Conference on Advanced Learning Technologies.

[23]  Hirokazu Kato,et al.  Augmented Reality Learning Experiences: Survey of Prototype Design and Evaluation , 2014, IEEE Transactions on Learning Technologies.

[24]  Ming-Kuan Tsai,et al.  Using augmented-reality-based mobile learning material in EFL English composition: An exploratory case study , 2013, Br. J. Educ. Technol..

[25]  Tsung-Yu Liu,et al.  A context-aware ubiquitous learning environment for language listening and speaking , 2009, J. Comput. Assist. Learn..

[26]  Yanjie Song,et al.  Analyzing students’ after-school artifact creation processes in a mobile-assisted language learning environment , 2011 .

[27]  Hiroaki Ogata,et al.  Computer supported ubiquitous learning environment for vocabulary learning , 2010, Int. J. Learn. Technol..

[28]  Chris Dede,et al.  EcoMOBILE: Integrating augmented reality and probeware with environmental education field trips , 2013, Comput. Educ..

[29]  F. N. Dempster,et al.  Effects of variable encoding and spaced presentations on vocabulary learning. , 1987 .

[30]  Lung-Hsiang Wong,et al.  Mobile-Assisted Vocabulary Learning in Real-Life Setting for Primary School Students: Two Case Studies , 2010, 2010 6th IEEE International Conference on Wireless, Mobile, and Ubiquitous Technologies in Education.

[31]  Carlos Delgado Kloos,et al.  Experimenting with electromagnetism using augmented reality: Impact on flow student experience and educational effectiveness , 2014, Comput. Educ..

[32]  Christian Sandor,et al.  Evaluating Augmented Reality for Situated Vocabulary Learning , 2014 .

[33]  Stephen R. Ellis,et al.  Visual clutter management in augmented reality: Effects of three label separation methods on spatial judgments , 2009, 2009 IEEE Symposium on 3D User Interfaces.

[34]  A. Collins,et al.  Situated Cognition and the Culture of Learning , 1989 .

[35]  Mau-Tsuen Yang,et al.  Computer-Assisted Culture Learning in an Online Augmented Reality Environment Based on Free-Hand Gesture Interaction , 2014, IEEE Transactions on Learning Technologies.

[36]  Christian Sandor,et al.  A usability scale for handheld augmented reality , 2014, VRST '14.

[37]  Maria Uther,et al.  Mobile Devices for Language Learning: Multimedia Approaches , 2009, Res. Pract. Technol. Enhanc. Learn..

[38]  Pierre Dillenbourg,et al.  Designing augmented reality for the classroom , 2013, Comput. Educ..

[39]  Chris Dede,et al.  Augmented Reality Teaching and Learning , 2014 .

[40]  Hiroaki Ogata,et al.  Context-aware support for computer-supported ubiquitous learning , 2004, The 2nd IEEE International Workshop on Wireless and Mobile Technologies in Education, 2004. Proceedings..

[41]  Takafumi Taketomi,et al.  Development of Handheld Augmented Reality X-Ray for K-12 Settings , 2015 .

[42]  Hiroaki Ogata,et al.  A Framework for Capturing, Sharing and Comparing Learning Experiences in a Ubiquitous Learning Environment , 2008, Res. Pract. Technol. Enhanc. Learn..

[43]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[44]  Tzung-Shi Chen,et al.  Context-Aware Writing in Ubiquitous Learning Environments , 2008, Fifth IEEE International Conference on Wireless, Mobile, and Ubiquitous Technology in Education (wmute 2008).

[45]  Hirokazu Kato,et al.  Toward Standard Usability Questionnaires for Handheld Augmented Reality , 2015, IEEE Computer Graphics and Applications.

[46]  Richard E. Mayer,et al.  The Cambridge Handbook of Multimedia Learning: Cognitive Theory of Multimedia Learning , 2005 .

[47]  Stefaan Ternier,et al.  Mobile Augmented Reality for Learning: A Case Study , 2011 .

[48]  Carlos Delgado Kloos,et al.  Impact of an augmented reality system on students' motivation for a visual art course , 2013, Comput. Educ..

[49]  Hsien-Sheng Hsiao,et al.  The Effects of Multimedia Annotations via PDA on EFL Learners' Vocabulary Learning , 2011, ICCE 2011.

[50]  S. Webb.  The Effects of Repetition on Vocabulary Knowledge , 2007 .

[51]  Sobah Abbas Petersen,et al.  Personalized and Contextualized Language Learning: Choose when, Where and What , 2009, Res. Pract. Technol. Enhanc. Learn..

[52]  Hirokazu Kato,et al.  Relation between location of information displayed by augmented reality and user's memorization , 2012, AH '12.

[53]  Dieter Schmalstieg,et al.  Image-driven view management for augmented reality browsers , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[54]  Kurt Squire,et al.  Environmental Detectives—the development of an augmented reality platform for environmental simulations , 2008 .

[55]  Jyh-Chong Liang,et al.  Current status, opportunities and challenges of augmented reality in education , 2013, Comput. Educ..

[56]  Takuji Narumi,et al.  Augmented reality flavors: gustatory display based on edible marker and cross-modal interaction , 2011, CHI.

[57]  Shanshan Li,et al.  Exploring the Potential for Augmented Reality to Motivate English Vocabulary Learning in Chinese College Students , 2014 .

[58]  Yukiko Watanabe,et al.  Key Aspects of Computer Assisted Vocabulary Learning (Cavl): Combined Effects of Media, Sequencing and Task Type , 2009, Res. Pract. Technol. Enhanc. Learn..

[59]  I-Jung Chen,et al.  Hypertext annotation: Effects of presentation formats and learner proficiency on reading comprehension and vocabulary learning in foreign languages , 2013, Comput. Educ..

[60]  Dayang Rohaya Awang Rambli,et al.  Preliminary Evaluation on User Acceptance of the Augmented Reality Use for Education , 2010, 2010 Second International Conference on Computer Engineering and Applications.

[61]  Margaret E. Morris,et al.  Context-Sensitive Microlearning of Foreign Language Vocabulary on a Mobile Device , 2007, AmI.

[62]  Eduardo E. Veas,et al.  Vesp’R: design and evaluation of a handheld AR device , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[63]  Jens Grubert,et al.  Augmented Reality Browser Survey , 2012 .

[64]  Henry Been-Lirn Duh,et al.  Influences of AR-Supported Simulation on Learning Effectiveness in Face-to-face Collaborative Learning for Physics , 2011, 2011 IEEE 11th International Conference on Advanced Learning Technologies.

[65]  Chih-Wei Chang,et al.  Improving Vocabulary Acquisition By Designing A Storytelling Robot , 2008, 2008 Eighth IEEE International Conference on Advanced Learning Technologies.

[66]  Rachid Benlamri,et al.  Context-Aware Services for Smart Learning Spaces , 2010, IEEE Transactions on Learning Technologies.

[67]  Chris Dede,et al.  Emerging Technologies, Ubiquitous Learning, and Educational Transformation , 2011, EC-TEL.

[68]  Dieter Schmalstieg,et al.  Sketching up the world: in situ authoring for mobile Augmented Reality , 2012, Personal and Ubiquitous Computing.

[69]  Richard E. Mayer,et al.  Multimedia Learning , 2001, Visible Learning Guide to Student Achievement.

[70]  Richard E. Mayer,et al.  Cognitive Theory of Multimedia Learning , 2021, The Cambridge Handbook of Multimedia Learning.