Mobile virtual reality for musical genre learning in primary education

Abstract Mobile virtual reality (VR) is increasingly becoming popular and accessible to everyone that holds a smartphone. In particular, digital didactics can take advantage of natural interaction and immersion in virtual environments, starting from primary education. This paper investigates the problem of enhancing music learning in primary education through the use of mobile VR. To this end, technical and methodological frameworks were developed, and were tested with two classes in the last year of a primary school (10 years old children). The classes were involved in an evaluation study on music genre identification and learning with a multi-platform mobile application called VR4EDU. Students were immersed in music performances of different genres (e.g., classical, country, jazz, and swing), navigating inside several musical rooms. The evaluation of the didactic protocol shows a statistically significant improvement in learning genre characterization (i.e., typical instruments and their spatial arrangements on stage) compared to traditional lessons with printed materials and passive listening. These results show that the use of mobile VR technologies in synergy with traditional teaching methodologies can improve the music learning experience in primary education, in terms of active listening, attention, and time. The inclusion of pupils with certified special needs strengthened our results.

[1]  Zane Stromberga,et al.  Virtualisation devices for student learning: Comparison between desktop-based (Oculus Rift) and mobile-based (Gear VR) virtual reality in medical and health science education , 2017 .

[2]  M. Christine Dwyer,et al.  Reinvesting in Arts Education: Winning America's Future through Creative Schools. , 2011 .

[3]  Rob Hamilton,et al.  Gesture-based Collaborative Virtual Reality Performance in Carillon , 2016, ICMC.

[4]  Federico Avanzini,et al.  Improving elevation perception with a tool for image-guided head-related transfer function selection , 2017 .

[5]  Thomas A. Furness,et al.  Virtual Reality and Education , 1994, Exploring a New Partnership: Children, Teachers and Technology.

[6]  Brian D. Simpson,et al.  Do you hear where I hear?: isolating the individualized sound localization cues , 2014, Front. Neurosci..

[7]  Kyle Johnsen,et al.  An accessible platform for everyday educational virtual reality , 2016, 2016 IEEE 2nd Workshop on Everyday Virtual Reality (WEVR).

[8]  Aaron Williamon,et al.  Musicians’ perceptions and experiences of using simulation training to develop performance skills , 2017 .

[9]  Inwook Hwang,et al.  AirPiano: Enhancing music playing experience in virtual reality with mid-air haptic feedback , 2017, 2017 IEEE World Haptics Conference (WHC).

[10]  Ann L. Brown,et al.  How people learn: Brain, mind, experience, and school. , 1999 .

[11]  Nolan Ung,et al.  Utilizing virtual and augmented reality for educational and clinical enhancements in neurosurgery , 2017, Journal of Clinical Neuroscience.

[12]  Eric D. Ragan,et al.  Supporting cognitive processing with spatial information presentations in virtual environments , 2012, Virtual Reality.

[13]  Chris Dede,et al.  Immersive Interfaces for Engagement and Learning , 2009, Science.

[14]  Barney Dalgarno,et al.  What are the learning affordances of 3-D virtual environments? , 2010, Br. J. Educ. Technol..

[15]  Frank Steinicke,et al.  Human Walking in Virtual Environments: Perception, Technology, and Applications , 2013 .

[16]  Z. Zacharia,et al.  Physical and Virtual Laboratories in Science and Engineering Education , 2013, Science.

[17]  E. Tulving,et al.  Effectiveness of retrieval cues in memory for words. , 1968, Journal of experimental psychology.

[18]  N. Burgess,et al.  Spatial memory: how egocentric and allocentric combine , 2006, Trends in Cognitive Sciences.

[19]  G. Riva,et al.  Virtual reality in anxiety disorders: the past and the future , 2008, Expert review of neurotherapeutics.

[20]  Christine Youngblut,et al.  Educational Uses of Virtual Reality Technology. , 1998 .

[21]  Mariano Alcañiz Raya,et al.  Affective Interactions Using Virtual Reality: The Link between Presence and Emotions , 2007, Cyberpsychology Behav. Soc. Netw..

[22]  Mel Ainscow,et al.  Developing inclusive education systems: the role of organisational cultures and leadership , 2010 .

[23]  Harry E. Price,et al.  Feasibility of Using an Augmented Immersive Virtual Reality Learning Environment to Enhance Music Conducting Skills , 2017 .

[24]  Karsten Bormann,et al.  Presence and the Utility of Audio Spatialization , 2005, Presence: Teleoperators & Virtual Environments.

[25]  Marc Conrad,et al.  Increasing student engagement through virtual interactions: How? , 2017, Virtual Reality.

[26]  Florent Berthaut,et al.  Spatial Interfaces and Interactive 3D Environments for Immersive Musical Performances , 2016, IEEE Computer Graphics and Applications.

[27]  Joseph Chazalon,et al.  Augmented songbook: an augmented reality educational application for raising music awareness , 2018, Multimedia Tools and Applications.

[28]  Cumhur Erkut,et al.  Virtual Reality Musical Instruments: State of the Art, Design Principles, and Future Directions , 2016, Computer Music Journal.

[29]  N. Cowan,et al.  Scope of attention, control of attention, and intelligence in children and adults , 2006, Memory & cognition.

[30]  Stefania Serafin,et al.  Considerations on the use of virtual and augmented reality technologies in music education , 2017, 2017 IEEE Virtual Reality Workshop on K-12 Embodied Learning through Virtual & Augmented Reality (KELVAR).

[31]  Federico Fontana,et al.  Detection of keyboard vibrations and effects on perceived piano quality. , 2017, The Journal of the Acoustical Society of America.

[32]  Christoph Hölscher,et al.  Taxonomy of Human Wayfinding Tasks: A Knowledge-Based Approach , 2009, Spatial Cogn. Comput..

[33]  Marc Leman,et al.  Action-based effects on music perception , 2013, Front. Psychol..

[34]  Mary C. Whitton,et al.  Walking > walking-in-place > flying, in virtual environments , 1999, SIGGRAPH.

[35]  Margot Prior,et al.  The Cognitive and Academic Benefits of Music to Children: Facts and fiction , 2006 .

[36]  N. Burgess,et al.  The Cognitive Architecture of Spatial Navigation: Hippocampal and Striatal Contributions , 2015, Neuron.

[37]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[38]  W. Trost,et al.  How musical training affects cognitive development: rhythm, reward and other modulating variables , 2014, Front. Neurosci..

[39]  Rebecca J Woods,et al.  Effects of Typeface and Font Size on Legibility for Children , 2005 .

[40]  Riccardo Scateni,et al.  RiftArt: Bringing Masterpieces in the Classroom through Immersive Virtual Reality , 2015, STAG.

[41]  Wijnand A. IJsselsteijn,et al.  Towards the Simplicity of Complex Interfaces: Applying Ephemeral Adaptation to Enhance User Performance and Satisfaction , 2017, Symbiotic.

[42]  Fernand Gobet,et al.  When the music's over. Does music skill transfer to children's and young adolescents' cognitive and academic skills? A meta-analysis , 2017 .

[43]  Burkhard Wünsche,et al.  Music Education using Augmented Reality with a Head Mounted Display , 2013, AUIC.

[44]  Eric D. Ragan,et al.  CI-Spy: Designing A Mobile Augmented Reality System for Scaffolding Historical Inquiry Learning , 2015, 2015 IEEE International Symposium on Mixed and Augmented Reality - Media, Art, Social Science, Humanities and Design.

[45]  Basheer Tome,et al.  Andante: Walking Figures on the Piano Keyboard to Visualize Musical Motion , 2014, NIME.

[46]  Steven M. Smith,et al.  Environmental context-dependent memory: A review and meta-analysis , 2001, Psychonomic bulletin & review.

[47]  Thomas F Münte,et al.  Unforgettable film music: The role of emotion in episodic long-term memory for music , 2008, BMC Neuroscience.

[48]  Baltasar Fernández-Manjón,et al.  Application of a low-cost web-based simulation to improve students' practical skills in medical education , 2010, Int. J. Medical Informatics.

[49]  J. Mandler,et al.  On the coding of spatial information , 1977, Memory & cognition.

[50]  Lely Adriana Luengas,et al.  Virtual flute: Electronic device that uses virtual reality to teach how to play a flute , 2010, IEEE EDUCON 2010 Conference.

[51]  Wolfgang Hürst,et al.  Mobile 3D graphics and virtual reality interaction , 2011, Advances in Computer Entertainment Technology.

[52]  Johanna Pirker,et al.  Evaluating Experiences in Different Virtual Reality Setups , 2016, ICEC.

[53]  Johanna Höysniemi,et al.  Proceedings of the 2006 conference on Interaction design and children , 2006 .

[54]  Zahira Merchant,et al.  The learner characteristics, features of desktop 3D virtual reality environments, and college chemistry instruction: A structural equation modeling analysis , 2012, Comput. Educ..

[55]  Zahira Merchant,et al.  Effectiveness of virtual reality-based instruction on students' learning outcomes in K-12 and higher education: A meta-analysis , 2014, Comput. Educ..

[56]  Mariano Alcañiz Raya,et al.  Learning in serious virtual worlds: Evaluation of learning effectiveness and appeal to students in the E-Junior project , 2010, Comput. Educ..

[57]  Ahmed Amer,et al.  Affordable altered perspectives: Making augmented and virtual reality technology accessible , 2014, IEEE Global Humanitarian Technology Conference (GHTC 2014).

[58]  George W. Fitzmaurice,et al.  Spotlight: directing users' attention on large displays , 2005, CHI.

[59]  Aliyah Nur Zafirah Sanusi,et al.  Go Virtual: Exploring Augmented Reality Application in Representation of Steel Architectural Construction for the Enhancement of Architecture Education , 2017 .

[60]  John B. Black,et al.  Direct manipulation is better than passive viewing for learning anatomy in a three-dimensional virtual reality environment , 2017, Comput. Educ..

[61]  Doo Hun Lim,et al.  Application of Virtual and Augmented Reality for Training and Mentoring of Higher Education Instructors , 2019, Handbook of Research on Virtual Training and Mentoring of Online Instructors.

[62]  Michael Vorlnder,et al.  Auralization: Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality , 2020 .

[63]  Xia Zheng,et al.  The use of virtual reality for episodic memory assessment: effects of active navigation. , 2011, Experimental psychology.

[64]  Hiroshi Ishii,et al.  Andantino: Teaching Children Piano with Projected Animated Characters , 2016, IDC.

[65]  Michele Geronazzo,et al.  Interactive spatial sonification for non-visual exploration of virtual maps , 2016, Int. J. Hum. Comput. Stud..

[66]  L. Jäncke.  Music, memory and emotion , 2008, Journal of biology.