M2S maps: supporting real-world navigation with mobile VR

Mobile devices are becoming increasingly integrated into our society. In addition to entertaining people with applications like pervasive games, mobile devices can also help to address cognitive challenges people face in the real world. This paper, by drawing on research findings from cognitive psychology and geography, explores a design to use mobile VR to help people overcome one cognitive barrier in navigation, which is to establish the correspondence between 2D spatial information found in maps and 3D entities they perceive from the real world. The design offers users multi-format, multi-scale, and semantic (M2S) maps, ranging from 2D maps to 3D immersive environments, and helps users to connect 2D maps to the real world through 3D environments which are equipped with semantic representation and animation techniques. Consequently, users can apply various kinds of spatial knowledge, 2D or 3D, in understanding the real world as well as assisting in navigation. This research enhances the design repertoire of mobile VR, and suggests a way to integrate virtual environments into people’s real-world life by examining the cognitive implications of 3D models on users’ activities.

[1]  Barbara Hayes-Roth,et al.  Differences in spatial knowledge acquired from maps and navigation , 1982, Cognitive Psychology.

[2]  Richard E. Parent,et al.  Computer animation - algorithms and techniques , 2012 .

[3]  Georg Schneider,et al.  Mobile Reality: A PDA-Based Multimodal Framework Synchronizing a Hybrid Tracking Solution with 3D Graphics and Location-Sensitive Speech Interaction , 2002, UbiComp.

[4]  Riku Suomela,et al.  The evolution of perspective view in WalkMap , 2003, Personal and Ubiquitous Computing.

[5]  J. Loomis,et al.  Visually Controlled Locomotion: Its Dependence on Optic Flow, Three-Dimensional Space Perception, and Cognition , 1998 .

[6]  Marion A. Eppler,et al.  Development of Visually Guided Locomotion , 1998 .

[7]  Barbara Tversky,et al.  Spatial Information Theory A Theoretical Basis for GIS , 1993, Lecture Notes in Computer Science.

[8]  Norman G. Vinson,et al.  Design guidelines for landmarks to support navigation in virtual environments , 1999, CHI '99.

[9]  Kevin Lynch,et al.  The Image of the City , 1960 .

[10]  J. Huttenlocher,et al.  Making Space: The Development of Spatial Representation and Reasoning , 2000 .

[11]  Enrico Puppo,et al.  Simplification, LOD and MultiresolutionPrinciples and Applications , 1997, Eurographics.

[12]  George W. Furnas,et al.  Navigation in electronic worlds: a CHI 97 workshop , 1997, SGCH.

[13]  T. McNamara,et al.  Subjective hierarchies in spatial memory. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[14]  Gary Marsden,et al.  Navigation control for mobile virtual environments , 2005, Mobile HCI.

[15]  B Hommel,et al.  Hierarchical coding in the perception and memory of spatial layouts , 2000, Psychological research.

[16]  Robert Headon,et al.  Movement Awareness for Ubiquitous Game Control , 2002, Personal and Ubiquitous Computing.

[17]  Andrew E. Johnson,et al.  CALVIN: an immersimedia design environment utilizing heterogeneous perspectives , 1996, Proceedings of the Third IEEE International Conference on Multimedia Computing and Systems.

[18]  Carl Machover,et al.  Virtual reality , 1994, IEEE Computer Graphics and Applications.

[19]  Rudy Darken,et al.  Navigating large virtual spaces , 1996, Int. J. Hum. Comput. Interact..

[20]  Alois Ferscha,et al.  Pervasive information acquisition for mobile AR-navigation systems , 2003, 2003 Proceedings Fifth IEEE Workshop on Mobile Computing Systems and Applications.

[21]  G W Evans,et al.  Cognitive mapping: knowledge of real-world distance and location information. , 1980, Journal of experimental psychology. Human learning and memory.

[22]  Alan Watt,et al.  3D games : animation and advanced real-time rendering , 2003 .

[23]  Carole Goble,et al.  The Montagues and the Capulets , 2004, Comparative and functional genomics.

[24]  Adrian David Cheok,et al.  Pervasive games: bringing computer entertainment back to the real world , 2005, CIE.

[25]  Mark Finn,et al.  The handheld classroom: educational implications of mobile computing , 2004 .

[26]  Randy Pausch,et al.  Virtual reality on a WIM: interactive worlds in miniature , 1995, CHI '95.

[27]  Dylan M. Jones,et al.  Navigating Buildings in "Desk-Top" Virtual Environments: Experimental Investigations Using Extended Navigational Experience , 1997 .

[28]  R. Malaka,et al.  DEEP MAP: Challenging IT Research In The Framework Of A Tourist Information System , 2000 .

[29]  Andreas Butz,et al.  The connected user interface: realizing a personal situated navigation service , 2004, IUI '04.

[30]  Morris Sloman,et al.  A framework for contextual mediation in mobile and ubiquitous computing applied to the context-aware adaptation of maps , 2004, Personal and Ubiquitous Computing.

[31]  F. J. Langdon,et al.  The Child's Conception of Space , 1967 .

[32]  Reginald G. Golledge,et al.  HUMAN WAYFINDING AND COGNITIVE MAPS , 2003 .

[33]  Frederico T. Fonseca,et al.  Play as the Way Out of the Newspeak-Tower of Babel Dilemma in Data Modeling , 2005, ICIS.

[34]  Bernhard Hommel,et al.  Acquisition of Cognitive Aspect Maps , 2003, Spatial Cognition.

[35]  Joseph J. LaViola,et al.  Hands-free multi-scale navigation in virtual environments , 2001, I3D '01.

[36]  John H. Bailey,et al.  Virtual spaces and real world places: transfer of route knowledge , 1996, Int. J. Hum. Comput. Stud..

[37]  David Kirsh,et al.  Worldlets—3D thumbnails for wayfinding in virtual environments , 1997, UIST '97.

[38]  Kay M. Stanney,et al.  A Theoretical Model of Wayfinding in Virtual Environments: Proposed Strategies for Navigational Aiding , 1999, Presence.

[39]  Albert C. Stevens,et al.  Distortions in judged spatial relations , 1978, Cognitive Psychology.

[40]  Randy F. Pausch,et al.  Navigation with place representations and visible landmarks , 2004, IEEE Virtual Reality 2004.

[41]  Steve Benford,et al.  Where on-line meets on the streets: experiences with mobile mixed reality games , 2003, CHI '03.

[42]  George W. Furnas,et al.  Navigation in Electronic Worlds. , 1997 .

[43]  R. Downs,et al.  Image and Environment: Cognitive Mapping and Spatial Behavior , 2017 .

[44]  Monika Sester,et al.  LANDMARKS FOR ROUTING - AUTOMATIC IDENTIFICATION, EXTRACTION AND VISUALIZATION , 2002 .

[45]  Heng Wang,et al.  TGH: a case study of designing natural interaction for mobile guide systems , 2005, Mobile HCI.

[46]  E. Tolman.  Cognitive maps in rats and men. , 1948, Psychological review.

[47]  George W. Furnas,et al.  mCVEs: Using Cross-Scale Collaboration to Support User Interaction with Multiscale Structures , 2005, Presence: Teleoperators & Virtual Environments.

[48]  Albrecht Schmidt,et al.  Advanced Interaction in Context , 1999, HUC.

[49]  Rudolph P. Darken,et al.  A Handheld Computer as an Interaction Device to a Virtual Environment , 1999 .

[50]  Bill N. Schilit,et al.  Context-aware computing applications , 1994, Workshop on Mobile Computing Systems and Applications.

[51]  Leo Bagrow,et al.  History of cartography , 1965 .

[52]  Rudy Darken,et al.  Map usage in virtual environments: orientation issues , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[53]  Gregory D. Abowd,et al.  Social Disclosure of Place: From Location Technology to Communication Practices , 2005, Pervasive.

[54]  Nigel Foreman,et al.  Transfer of Spatial Information from a Virtual to a Real Environment , 1997, Hum. Factors.

[55]  Tom Rodden,et al.  Exploiting Context in HCI Design for Mobile Systems , 1998 .

[56]  D. Levine,et al.  Two visual systems in mental imagery , 1985, Neurology.

[57]  Christian Freksa,et al.  Cognitive Requirements on Making and Interpreting Maps , 1997, COSIT.

[58]  Ismo Rakkolainen,et al.  A 3D City Info for mobile users , 2001, Comput. Graph..

[59]  Stephen A. Brewster,et al.  Using Landmarks to Support Older People in Navigation , 2004, Mobile HCI.

[60]  Wolfgang Wahlster,et al.  A resource-adaptive mobile navigation system , 2002, IUI '02.

[61]  Manfred Tscheligi,et al.  Landmarks: yes; but which?: five methods to select optimal landmarks for a landmark- and speech-based guiding system , 2005, Mobile HCI.

[62]  Andrew J. May,et al.  Pedestrian navigation aids: information requirements and design implications , 2003, Personal and Ubiquitous Computing.

[63]  Paolo Cignoni,et al.  Multiresolution Representation and Visualization of Volume Data , 1997, IEEE Trans. Vis. Comput. Graph..

[64]  Josephine Reid,et al.  "Fancy a schmink?": a novel networked game in a café , 2004, ACE '04.

[65]  J. Loomis,et al.  Visuallv Controlled Locomotion: Its ~e$endence on Optic Flow, ~hree-~imensional Space Perception, and Cognition , 1998 .

[66]  Wilfried Brauer,et al.  Spatial cognition III: Routes and navigation, human memory and learning, spatial representation and spatial learning , 2003 .

[67]  Luca Chittaro,et al.  Location-aware visualization of VRML models in GPS-based mobile guides , 2005, Web3D '05.

[68]  Stephan Winter,et al.  Enriching Wayfinding Instructions with Local Landmarks , 2002, GIScience.

[69]  Alois Ferscha,et al.  A New Visualization Concept for Navigation Systems , 2004, User Interfaces for All.

[70]  Sabine Timpf,et al.  Granularity Transformations in Wayfinding , 2003, Spatial Cognition.

[71]  Gregory D. Abowd,et al.  Towards a Better Understanding of Context and Context-Awareness , 1999, HUC.

[72]  R. Hart,et al.  The Development of Spatial Cognition: A Review. , 1973 .

[73]  Midori A. Harris,et al.  Standards and Ontologies for Functional Genomics 2 , 2004, Comparative and functional genomics.

[74]  Kori Inkpen Quinn,et al.  Effect of location-awareness on rendezvous behaviour , 2005, CHI EA '05.

[75]  Christian Kray,et al.  Presenting route instructions on mobile devices , 2003, IUI '03.