Effects of position of real-time translation on AR glasses

Augmented reality (AR) provides users with contextually relevant multimedia content by overlaying it on real-world objects. However, overlaying virtual content on real-world objects can cause occlusion. Especially for learning use cases, the occlusion might result in missing real-world information important for learning gain. Therefore, it is important to understand how virtual content should be positioned relative to the related real-world information without negatively affecting the learning experience. Thus, we conducted a study with 12 participants using AR glasses to investigate the position of virtual content using a vocabulary learning task. Participants learned foreign words shown in the surrounding while viewing translations using AR glasses as an overlay, on the right or below the foreign word. We found that showing virtual translations on top of foreign words significantly decreases comprehension and increase users' task load. Insights from our study inform the design of applications for AR glasses supporting vocabulary learning.

[1]  Shengdong Zhao,et al.  Positioning Glass: Investigating Display Positions of Monocular Optical See-Through Head-Mounted Display , 2016, ChineseCHI2016.

[2]  Ahmad Afif Supianto,et al.  BandoAR: Real-Time Text Based Detection System Using Augmented Reality for Media Translator Banjar Language to Indonesian with Smartphone , 2018, 2018 IEEE 5th International Conference on Engineering Technologies and Applied Sciences (ICETAS).

[3]  Kevin White,et al.  Augmented reality in education and training: pedagogical approaches and illustrative case studies , 2018, J. Ambient Intell. Humaniz. Comput..

[4]  Karim Sadeghi,et al.  Effects of computer-assisted glosses on EFL learners’ vocabulary acquisition and reading comprehension in three learning conditions , 2017 .

[5]  R. K. Johnson,et al.  Vocabulary Learning Strategies and Language Learning Outcomes , 1996 .

[6]  Matthew Turk,et al.  TranslatAR: A mobile augmented reality translator , 2011, 2011 IEEE Workshop on Applications of Computer Vision (WACV).

[7]  Tsutomu Terada,et al.  An information layout method for an optical see-through head mounted display focusing on the viewability , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[8]  Yi-Hui Chiu,et al.  Computer-assisted second language vocabulary instruction: A meta-analysis , 2013, Br. J. Educ. Technol..

[9]  Christian Sandor,et al.  Evaluating Augmented Reality for Situated Vocabulary Learning , 2014 .

[10]  Ali Farhan AbuSeileek Hypermedia Annotation Presentation: Learners' Preferences and Effect on EFL Reading Comprehension and Vocabulary Acquisition. , 2008 .

[11]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[12]  Hirokazu Kato,et al.  Relation between location of information displayed by augmented reality and user's memorization , 2012, AH '12.

[13]  Tobias Höllerer,et al.  ARbis Pictus: A Study of Vocabulary Learning with Augmented Reality , 2017, IEEE Transactions on Visualization and Computer Graphics.

[14]  Ramu Parupalli,et al.  Mobile augmented reality application for Telugu language learning , 2014, 2014 IEEE International Conference on MOOC, Innovation and Technology in Education (MITE).

[15]  Lap-Kei Lee,et al.  A Mobile Game for Learning English Vocabulary with Augmented Reality Block Builder , 2019, Communications in Computer and Information Science.

[16]  M. Yoshii L1 and L2 Glosses: Their Effects on Incidental Vocabulary Learning , 2006 .

[17]  Mari-Pauliina Vainikainen,et al.  Making the invisible observable by Augmented Reality in informal science education context , 2017 .

[18]  James T. Miller,et al.  An Empirical Evaluation of the System Usability Scale , 2008, Int. J. Hum. Comput. Interact..

[19]  Kangdon Lee,et al.  Augmented Reality in Education and Training , 2012, TechTrends.

[20]  Hirokazu Kato,et al.  Augmented reality as multimedia: the case for situated vocabulary learning , 2016, Res. Pract. Technol. Enhanc. Learn..

[21]  Kiyoshi Kiyokawa,et al.  Managing mobile text in head mounted displays: studies on visual preference and text placement , 2014, MOCO.

[22]  Juan Garzón,et al.  Meta-analysis of the impact of Augmented Reality on students’ learning gains , 2019, Educational Research Review.

[23]  Steffen Gauglitz,et al.  TranslatAR : A Mobile Augmented Reality Translator on the Nokia N 900 , 2009 .

[24]  Stefan Decker,et al.  Integrating Text with Video and 3D Graphics: The Effects of Text Drawing Styles on Text Readability , 2010, CHI.

[25]  Borko Furht,et al.  Augmented Reality: An Overview , 2011, Handbook of Augmented Reality.

[26]  Christopher Wasko,et al.  What Teachers Need to Know About Augmented Reality Enhanced Learning Environments , 2013 .

[27]  Emrah Baki A COMPARISON OF UNDERGRADUATE STUDENTS' ENGLISH VOCABULARY LEARNING: USING MOBILE PHONES AND FLASH CARDS , 2010 .

[28]  Jun Du,et al.  Snap and Translate Using Windows Phone , 2011, 2011 International Conference on Document Analysis and Recognition.

[29]  Wernhuar Tarng,et al.  A Study of Campus Butterfly Ecology Learning System Based on Augmented Reality and Mobile Learning , 2012, 2012 IEEE Seventh International Conference on Wireless, Mobile and Ubiquitous Technology in Education.

[30]  Kan Kan Chan,et al.  Using Augmented Reality Flashcards to Learn Vocabulary in Early Childhood Education , 2019, Journal of Educational Computing Research.

[31]  Ann McNamara,et al.  Enhancing art history education through mobile augmented reality , 2011, VRCAI.

[32]  Mark Billinghurst,et al.  Augmented Reality in the Classroom , 2012, Computer.

[33]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[34]  Andreas Dengel,et al.  User attention oriented augmented reality on documents with document dependent dynamic overlay , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[35]  Deborah Hix,et al.  The Effects of Text Drawing Styles, Background Textures, and Natural Lighting on Text Legibility in Outdoor Augmented Reality , 2006, Presence: Teleoperators & Virtual Environments.

[36]  BillinghurstMark,et al.  The MagicBookMoving Seamlessly between Reality and Virtuality , 2001 .

[37]  F Muni,et al.  Hypermedia Annotation Presentation: Learners' Preferences and Effect on EFL Reading Comprehension and Vocabulary Acquisition , 2008 .

[38]  Bulent Tarman,et al.  A Model to Explore Turkish Teachers' ICT Integration Stages. , 2010 .

[39]  Iulian Radu,et al.  Augmented reality in education: a meta-review and cross-media analysis , 2014, Personal and Ubiquitous Computing.

[40]  David E. Breen,et al.  Interactive Occlusion and Collision of Real and Virtual Objects in Augmented Reality , 2000 .

[41]  Chin-Siang Ang,et al.  Investigating the use of mobile applications in everyday language learning , 2017, Int. J. Mob. Learn. Organisation.

[42]  Daniel Sonntag,et al.  A mixed reality head-mounted text translation system using eye gaze input , 2014, IUI.

[43]  Dorothy M. Chun,et al.  Effects of Multimedia Annotations on Vocabulary Acquisition , 1996 .

[44]  Niels Henze,et al.  Reading on Smart Glasses: The Effect of Text Position, Presentation Type and Walking , 2018, CHI.

[45]  Yu-Chien Chen,et al.  A study of comparing the use of augmented reality and physical models in chemistry education , 2006, VRCIA '06.

[46]  Matt Bower,et al.  Augmented reality in Education — Cases, places, and potentials , 2013, 2013 IEEE 63rd Annual Conference International Council for Education Media (ICEM).

[47]  Ivan Poupyrev,et al.  The MagicBook - Moving Seamlessly between Reality and Virtuality , 2001, IEEE Computer Graphics and Applications.

[48]  Sandra G. Hart,et al.  Nasa-Task Load Index (NASA-TLX); 20 Years Later , 2006 .

[49]  Hirokazu Kato,et al.  Relationship between features of augmented reality and user memorization , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[50]  Robert Godwin-Jones Augmented reality and language learning: From annotated vocabulary to place-based mobile games , 2016 .

[51]  Dieter Schmalstieg,et al.  Mathematics and geometry education with collaborative augmented reality , 2003, Comput. Graph..

[52]  Shih-Yeh Chen,et al.  A Study on Integrating Augmented Reality Technology and Game-Based Learning Model to Improve Motivation and Effectiveness of Learning English Vocabulary , 2018, 2018 1st International Cognitive Cities Conference (IC3).

[53]  Olga Scrivner,et al.  Augmented reality digital technologies (ARDT) for foreign language teaching and learning , 2016, 2016 Future Technologies Conference (FTC).