Linking Audio And Visual Information While Navigating In A Virtual Reality Kiosk Display

3D interactive virtual reality museum exhibits should be easy to use, entertaining, and informative. If the interface is intuitive, it will allow the user more time to learn the educational content of the exhibit. This research deals with interface issues concerning activating audio descriptions of images in such exhibits while the user is navigating. Five methods for activating audio descriptions were implemented and evaluated to find the most effective. These range roughly on a passive-active continuum. With the more passive methods, an audio explanation was triggered by simple proximity to an image of interest. The more active methods involved users orienting themselves and pressing a button to start the audio. In the most elaborate method, once the visitor had pressed a trigger button, the system initiated a "tractor-beam" that animated the viewpoint to a location in front of, and facing, the image of interest before starting the audio. The results of this research suggest that the more active methods were both preferred and more effective in getting visitors to face objects of interest while audio played. The tractor-beam method was best overall and implemented in a museum exhibit. ********** Modern computer technology has made possible 3D interactive public kiosks that provide the user with a multi-media rich environment that may include text, graphics, images, sound-clips, video, and animations. Often these environments allow the user to interactively select content and navigate through the 3D space to retrieve information, however, the navigation task may distract the user from this information. Ideally, the user should enjoy the benefits of these kiosks without sacrificing the ability to acquire the information they contain. Developing these types of interactive environments is a complex task due to the specific requirements of kiosks. They should be exceptionally easy to use, as they must be proficiently operated within a few minutes; they should be self-explanatory, as there are no human helpers to interact with; and they should engage users with interesting content, so their experience will be a memorable one. This article is concerned with the particular problems associated with 3D interactive public kiosks; in particular, effectively linking visual 3D images with recorded spoken descriptions while navigating. Multimedia, cognitive, and learning theories suggest that the cognitive load placed on users by aspects of the kiosk, those not needed for learning the educational content, should be minimized (Schaller & Allison-Bunnell, 2003; Travis, Watson, & Atyeo, 1994). This requires finding an appropriate method for activating audio descriptions that is simple to learn and use. This research also had a practical goal. By obtaining a contract from the New Hampshire Seacoast Science Center, it was possible to design and build the interface for a 3D kiosk; with the intent to inform the public about aspects of the marine environment. The Seacoast Science Center preferred it to be a stereoscopic computer display with a fly-through interface, and wanted the main content to consist of video and still images distributed through the 3D environment. The challenge was to develop a technique enabling users to, on their own, make audio-visual connections easily, quickly, and naturally, without hindering their ability to navigate around the virtual environment. LITERATURE REVIEW There are many areas of prior research relevant to issues dealing with 3D virtual kiosks. Some of these include cognitive theories of how people learn, and theories that have been developed to account for why multimedia presentations can be more effective. In addition, there are studies of how to control the user's attention; studies relating to the best way of connecting images with audio while navigating; and studies of whether active learning environments are better than passive learning environments. …

[1]  M. D’Esposito.  Working memory. , 2008, Handbook of clinical neurology.

[2]  A. Feldman,et al.  The effect of active versus passive exploration on memory for spatial location in children. , 1979, Child development.

[3]  Anne H. Soukhanov,et al.  The american heritage dictionary of the english language , 1992 .

[4]  David H. Jonassen,et al.  Handbook of Research for educational Communications and Technology , 1997 .

[5]  D. Kolb.  Experiential Learning: Experience as the Source of Learning and Development , 1983 .

[6]  D. Jonassen,et al.  Activity theory as a framework for designing constructivist learning environments , 1999 .

[7]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .

[8]  Richard E. Mayer,et al.  A Cognitive Theory of Multimedia Learning: Implications for Design Principles , 2001 .

[9]  P. Wilson.  Active Exploration of a Virtual Environment Does Not Promote Orientation or Memory for Objects , 1999 .

[10]  E. Not,et al.  Intelligent Interactive Information Presentation for Cultural Tourism , 2005 .

[11]  G. Evans,et al.  Environment and Behavior , 2001 .

[12]  O. R. Roosenschoon,et al.  An interactive virtual soil museum: a pilot project of the International Soil Reference and Information Centre with focus on arid soils , 2002 .

[13]  F. P. Brooks,et al.  Grasping reality through illusion—interactive graphics serving science , 1988, CHI '88.

[14]  A. Collins,et al.  Situated Cognition and the Culture of Learning , 1989 .

[15]  James C. Lester,et al.  Animated Pedagogical Agents: Face-to-Face Interaction in Interactive Learning Environments , 2000 .

[16]  R. Cohen,et al.  The effect of barriers on spatial representations. , 1981, Child development.

[17]  R. Mayer,et al.  For whom is a picture worth a thousand words? Extensions of a dual-coding theory of multimedia learning. , 1994 .

[18]  Arthur I. Karshmer,et al.  Proceedings of the third international ACM conference on Assistive technologies , 1998 .

[19]  A. Paivio,et al.  Dual coding theory and education , 1991 .

[20]  David T. Schaller,et al.  Practicing What We Teach: How Learning Theory Can Guide Development Of Online ( Educational Activities , 2003 .

[21]  John T. Kelso,et al.  Effects of Active Exploration and Passive Observation on Spatial Learning in a CAVE , 2002 .

[22]  William Morris.  The American Heritage dictionary of the English language , 1969 .

[23]  Antonella De Angeli,et al.  Integration and synchronization of input modes during multimodal human-computer interaction , 1997, CHI.

[24]  James O. Carey,et al.  The systematic design of instruction , 1978 .

[25]  Nigel Foreman,et al.  Active Versus Passive Processing of Spatial Information in a Computer-Simulated Environment , 1997 .

[26]  W. Hibbard,et al.  Interactivity is the key , 1989, VVS '89.

[27]  R. Mayer,et al.  A Split-Attention Effect in Multimedia Learning: Evidence for Dual Processing Systems in Working Memory , 1998 .

[28]  Franca Garzotto,et al.  From Dust to Stardust: A Collaborative 3D Virtual Museum of Computer Science , 2001, ICHIM.

[29]  R. Mayer,et al.  Maximizing Constructivist Learning From Multimedia Communications by Minimizing Cognitive Load , 1999 .

[30]  Harold D. Fishbein,et al.  Learners' Questions and Comprehension in a Tutoring Setting. , 1990 .

[31]  Etienne Wenger,et al.  Situated Learning: Legitimate Peripheral Participation , 1991 .

[32]  M. Corbetta,et al.  Control of goal-directed and stimulus-driven attention in the brain , 2002, Nature Reviews Neuroscience.

[33]  Ben Shneiderman,et al.  Engagement Theory: A Framework for Technology-Based Teaching and Learning. , 1998 .

[34]  Ken Hinckley,et al.  A survey of design issues in spatial input , 1994, UIST '94.

[35]  David H. Jonassen,et al.  Constructivism: Implications for the Design and Delivery of Instruction , 1996 .

[36]  Qiming Wang,et al.  Making VRML accessible for people with disabilities , 1998, Assets '98.

[37]  Colin Ware,et al.  GeoZui3D: data fusion for interpreting oceanographic data , 2001, MTS/IEEE Oceans 2001. An Ocean Odyssey. Conference Proceedings (IEEE Cat. No.01CH37295).

[38]  Nancy Hazen,et al.  Spatial Exploration and Spatial Knowledge: Individual and Developmental Differences in Very Young Children. , 1982 .

[39]  R. Mayer,et al.  The instructive animation: helping students build connections between words and pictures in multimedia learning , 1992 .

[40]  Lindsay William Macdonald,et al.  Interacting with Virtual Environments , 1993 .

[41]  Ipke Wachsmuth,et al.  Deixis in Multimodal Human Computer Interaction: An Interdisciplinary Approach , 2003, Gesture Workshop.