The UW Virtual Brain Project: An immersive approach to teaching functional neuroanatomy

Learning functional neuroanatomy requires forming mental representations of 3D structure, but forming such representations from 2D textbook diagrams can be challenging. We address this challenge in the UW Virtual Brain Project by developing 3D narrated diagrams, which are interactive, guided tours through 3D models of perceptual systems. Lessons can be experienced in virtual realty (VR) or on a personal computer monitor (PC). We predicted participants would learn from lessons presented on both VR and PC devices (comparing pre-test/post-test scores), but that VR would be more effective for achieving both content-based learning outcomes (i.e test performance) and experience-based learning outcomes (i.e., reported enjoyment and ease of use). All participants received lessons about the visual system and auditory system, one in VR and one on a PC (order counterbalanced). We assessed content learning using a drawing/labeling task on paper (2D drawing) in Experiment 1 and a Looking Glass autostereoscopic display (3D drawing) in Experiment 2. In both experiments, we found that the UW Virtual Brain Project lessons were effective for teaching functional neuroanatomy, with no difference between devices. However, participants reported VR was more enjoyable and easier to use. We also evaluated the VR lessons in our Classroom Implementation during an undergraduate course on perception. Students reported that the VR lessons helped them make progress on course learning outcomes, especially for learning system pathways. They suggested lessons could be improved by adding more examples and providing more time to explore in VR. Public Significance Statement. We designed and evaluated interactive 3D narrated diagrams to teach functional neuroanatomy. These lessons can be experienced on desktop PCs and in virtual reality (VR), and are helpful for teaching undergraduates about structure and function of perceptual systems in the human brain. To learn functional anatomy, such as how sensory information is processed in the human brain, students must form mental representations of 3D anatomical structures. Evidence suggests forming mental representations is easier for learners when they are presented with 3D models (i.e., different views can be rendered by translation and rotation), compared with 2D images (see Yammine and Violato (2015) for a meta-analysis). This benefit of 3D models, at least in part, is because piecing together multiple views from 2D images incurs a cognitive load that detracts from learning the content, especially for learners with lower visual-spatial ability (Bogomolova et al., 2020; Cui et al., 2016). Prior studies have suggested physical models are better than computer models for illustrating gross anatomy (Khot et al., 2013; Preece et al., 2013; Wainman et al., 2020; Wainman et al., 2018). However, physical models are limited in their potential to illustrate dynamic, functional processes, such as how neural signals are triggered by sensory input and propagate through a perceptual system. Given that our focus is on functional anatomy, we will focus our discussion on computer-based models only. The present study is part of the UW Virtual Brain Project, in which we have developed and assessed a new approach for teaching students about functional *Correspondence concerning this article should be addressed to Karen Schloss, University of Wisconsin-Madison, 330 North Orchard Street, Room 3178 Madison, WI 53715. E-mail: kschloss@wisc.edu 1 ar X iv :s ub m it/ 38 96 67 5 [ cs .H C ] 3 0 A ug 2 02 1 The UW Virtual Brain Project • February 2021 • Preprint anatomy of perceptual pathways. Previous computerbased 3D models of the human brain were geared toward teaching medical students about gross anatomy (Adams & Wilson, 2011; L. K. Allen et al., 2016; Cui et al., 2017; Drapkin et al., 2015; Ekstrand et al., 2018; Kockro et al., 2015; Stepan et al., 2017).1 In contrast, our lessons give learners guided, first-person view tours through “3D narrated diagrams” illustrating the functional anatomy of the human brain. We use the term “3D narrated diagram” to refer to 3D models combined with labels and verbal descriptions, analogous to content found in textbook diagrams with corresponding text. They can also include animations that illustrate dynamic aspects of the system. Thus, 3D models form the basis for the environment used to teach students about sensory input, system pathways, and system purposes, which are key learning outcomes in an undergraduate course on sensation and perception. Our aim was to develop structured, self-contained lessons for an undergraduate audience, which harnessed principles for effective multimedia learning (Mayer, 2009). These principles have previously been shown to facilitate learning in a variety of domains. For example, using visual cues to signal students where to look during a lesson can help them learn about neural structures (signaling principle) (Jamet et al., 2008). Learners benefit from having self-paced controls through a lesson, compared with experiencing system-paced continuous animation (segmenting principle) (Hasler et al., 2007). And, receiving input from multiple modalities (audio narration plus visual illustration) can be better than receiving visual input alone (modality principle) (Harskamp et al., 2007). The UW Virtual Brain 3D narrated diagrams can be viewed on personal computer monitors (referred to as “PC”; the same image is presented to both eyes) or in virtual reality using a head mounted display (HMD) with stereoscopic depth (referred to as “VR”; different images are presented to each eye 2). In the VR version, the brain is room-sized, so learners can “immerse” their whole body inside the brain. This study investigated whether students made significant gains in content-based learning outcomes from the Virtual Brain lessons, and whether viewing device (VR vs. PC) influenced the degree to which learners achieved content-based and experience-based learning outcomes. Content-based learning outcomes included being able to describe (draw/label) key brain regions and pathways involved in processing visual and auditory input. Experience-based learning outcomes included finding the lessons enjoyable and easy to use. We predicted that learners would make significant gains in content-based learning outcomes from lessons experienced in both VR and PC viewing (compared to a pre-test baseline), but VR viewing would be more effective. We also predicted VR would be more effective for achieving experience-based learning outcomes. Previous work strongly supports our prediction for experience-based learning outcomes, demonstrating that VR facilitates enjoyment, engagement, and motivation, compared with less immersive experiences (Hu-Au & Lee, 2017; Makransky et al., 2020; Pantelidis, 2010; Parong & Mayer, 2018; Stepan et al., 2017). However, prior evidence concerning our prediction that VR would better support content-based learning outcomes is mixed. Research on learning 3D structure and spatial layout suggests VR should facilitate learning, but research on narrated lessons suggests VR may hinder learning, as discussed below. Research on learning 3D anatomical structure suggests stereoscopic viewing facilitates learning compared to monoscopic viewing of the same models, at least when viewing is interactive. A meta-analysis reported that viewing interactive stereoscopic 3D models provided a significant benefit, compared with viewing interactive monoscopic 3D models (i.e., the same image was presented to both eyes, or the image was presented to one eye only) (Bogomolova et al., 2020). For example, Wainman et al. (2020) found students learned better when stereoscopically viewing 3D models compared to when one eye was covered while using a VR HMD. The additional depth information provided by stereopsis likely contributes to these enhanced learning outcomes (Bogomolova et al., 2020; Wainman et al., 2020). Evidence suggests that stereoscopic information is especially beneficial for 3D perception under interactive viewing conditions where head trackingbased motion parallax information and task feedback are available (Fulvio & Rokers, 2017), perhaps because viewers tend to discount stereoscopic information under passive viewing conditions (Fulvio et al., 2020). This may explain why the contribution of stereopsis to achieve learning outcomes was more limited under passive viewing (Al-Khalili & Coppoc, 2014) and fixed viewpoint rendering (Chen et al., 2012; Luursema et al., 2008). A separate line of studies testing the ability to remember spatial layout in new environments suggests 1Studies evaluating the efficacy of these 3D models used a myriad of comparison conditions that differed from the 3D models in multiple dimensions. Thus, it challenging to form general conclusions from their results (see Wainman et al. (2020) for a discussion of this issue). 2We note earlier literature used the term “VR” in reference to viewing 3D models on 2D displays (e.g., computer monitors), rather than immersive head mounted displays (see Wainman et al. (2020) for a discussion of this issue). In this article, we reserve the term “VR” for head mounted displays, like an Oculus Rift, Oculus Go, or HTC Vive.