A dialogue based mobile virtual assistant for tourists: The SpaceBook Project

Abstract Ubiquitous mobile computing offers innovative approaches in the delivery of information that can facilitate free roaming of the city, informing and guiding the tourist as the city unfolds before them. However making frequent visual reference to mobile devices can be distracting, the user having to interact via a small screen thus disrupting the explorative experience. This research reports on an EU funded project, SpaceBook, that explored the utility of a hands-free, eyes-free virtual tour guide, that could answer questions through a spoken dialogue user interface and notify the user of interesting features in view while guiding the tourist to various destinations. Visibility modelling was carried out in real-time based on a LiDAR sourced digital surface model, fused with a variety of map and crowd sourced datasets (e.g. Ordnance Survey, OpenStreetMap, Flickr, Foursquare) to establish the most interesting landmarks visible from the user's location at any given moment. A number of variations of the SpaceBook system were trialled in Edinburgh (Scotland). The research highlighted the pleasure derived from this novel form of interaction and revealed the complexity of prioritising route guidance instruction alongside identification, description and embellishment of landmark information – there being a delicate balance between the level of information ‘pushed’ to the user, and the user's requests for further information. Among a number of challenges, were issues regarding the fidelity of spatial data and positioning information required for pedestrian based systems – the pedestrian having much greater freedom of movement than vehicles.

[1]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[2]  Tiphaine Dalmas,et al.  Learning foci for Question Answering over Topic Maps , 2009, ACL.

[3]  Fabio Dovis,et al.  Performance of the Proposed Galileo CBOC Modulation in Heavy Multipath Environment , 2008, Wirel. Pers. Commun..

[4]  Mark Billinghurst,et al.  Evaluating Augmented Reality Systems , 2011, Handbook of Augmented Reality.

[5]  Keith Cheverst,et al.  'Caches in the air': disseminating tourist information in the GUIDE system , 1999, Proceedings WMCSA'99. Second IEEE Workshop on Mobile Computing Systems and Applications.

[6]  Fredrik Espinoza,et al.  GeoNotes: Social and Navigational Aspects of Location-Based Information Systems , 2001, UbiComp.

[7]  Dan Roth,et al.  Learning Question Classifiers , 2002, COLING.

[8]  Max J. Egenhofer,et al.  Spatial Information Appliances: A Next Generation of Geographic Information Systems , 1999 .

[9]  Alois Ferscha,et al.  Augmented reality navigation systems , 2006, Universal Access in the Information Society.

[10]  Niels Henze,et al.  Tactile wayfinder: a non-visual support system for wayfinding , 2008, NordiCHI.

[11]  Lars Kulik,et al.  Location privacy and location-aware computing , 2006 .

[12]  Oliver Lemon,et al.  Multi-threaded Interaction Management for Dynamic Spatial Applications , 2014, DM@EACL.

[13]  Gary Priestnall,et al.  Mobile capture of remote points of interest using line of sight modelling , 2013, Comput. Geosci..

[14]  Oliver Lemon,et al.  Integrating Location, Visibility, and Question-Answering in a Spoken Dialogue System for Pedestrian City Exploration , 2012, SIGDIAL Conference.

[15]  Roberta L. Klatzky,et al.  Navigation System for the Blind: Auditory Display Modes and Guidance , 1998, Presence.

[16]  Sven Fuhrmann,et al.  Natural Conversational Interfaces to Geospatial Databases , 2005, Trans. GIS.

[17]  Steve Benford,et al.  Can you see me now? , 2006, TCHI.

[18]  Roberta L. Klatzky,et al.  A Geographical Information System for a GPS Based Personal Guidance System , 1998, Int. J. Geogr. Inf. Sci..

[19]  Stephen A. Brewster,et al.  The design of sonically-enhanced widgets , 1998, Interact. Comput..

[20]  Junjun Yin,et al.  Mobile Visibility Querying for LBS , 2010, Trans. GIS.

[21]  Jaewoo Chung,et al.  Mindful navigation for pedestrians: Improving engagement with augmented reality , 2016 .

[22]  Stephan Winter,et al.  Including landmarks in routing instructions , 2010, J. Locat. Based Serv..

[23]  Jennifer C. Lai,et al.  Conversational interfaces , 2000, CACM.

[24]  Stephen Brewster,et al.  Using non-speech sound to overcome information overload , 1997 .

[25]  Reginald G. Golledge,et al.  Place recognition and wayfinding: Making sense of space , 1992 .

[26]  Rui Gu,et al.  Research on the Key Techniques of Augmented Reality Navigation , 2016 .

[27]  Georg Gartner,et al.  A critical evaluation of location based services and their potential , 2007, J. Locat. Based Serv..

[28]  Stephen Clark,et al.  A New Corpus and Imitation Learning Framework for Context-Dependent Semantic Parsing , 2014, TACL.

[29]  Ratnadeep R. Deshmukh,et al.  A Review on Different Approaches for Speech Recognition System , 2015 .

[30]  Yong Chen,et al.  Using Trust for Secure Collaboration in Uncertain Environments , 2003, IEEE Pervasive Comput..

[31]  Geoffrey Yu-Kai Shea,et al.  EASYGO - A public transport query and guiding LBS , 2007, Location Based Services and TeleCartography.

[32]  Andrew J. May,et al.  Pedestrian navigation aids: information requirements and design implications , 2003, Personal and Ubiquitous Computing.

[33]  Daniel R. Montello,et al.  Scale and Multiple Psychologies of Space , 1993, COSIT.

[34]  Didier Chincholle,et al.  Lost or Found? A Usability Evaluation of a Mobile Navigation and Location-Based Service , 2002, Mobile HCI.

[35]  William A. Mackaness,et al.  Development of a Speech-Based Augmented Reality System to Support Exploration of Cityscape , 2006, Trans. GIS.

[36]  Stephen C. Hirtle,et al.  The Nature of Landmarks for Real and Electronic Spaces , 1999, COSIT.

[37]  Mirella Lapata,et al.  WikiSimple: Automatic Simplification of Wikipedia Articles , 2011, AAAI.

[38]  Oliver Lemon,et al.  multithreaded context for robust conversational interfaces: Context-sensitive speech recognition and interpretation of corrective fragments , 2004, TCHI.

[39]  Sue Long,et al.  Cyberguide: prototyping context-aware mobile applications , 1996, CHI 1996.

[40]  John M. Carroll,et al.  Designing Interaction: Psychology at the Human-Computer Interface , 1991 .

[41]  J. F. Kelley,et al.  An empirical methodology for writing user-friendly natural language computer applications , 1983, CHI '83.

[42]  Wu Chen,et al.  Improving Integrity and Reliability of Map Matching Techniques , 2006 .

[43]  Scott A. Bridwell.  The dimensions of locational privacy , 2007 .

[44]  A. May,et al.  Incorporating Landmarks in Driver Navigation System Design: An Overview of Results from the REGIONAL Project , 2005, Journal of Navigation.

[45]  Mattias Wallergård,et al.  WozARd: A Wizard of Oz Method for Wearable Augmented Reality Interaction - A Pilot Study , 2015, Adv. Hum. Comput. Interact..

[46]  Simon Kingham,et al.  Advancing visibility modelling algorithms for urban environments , 2010, Comput. Environ. Urban Syst..

[47]  Michael F. McTear,et al.  Book Review: Spoken Dialogue Technology: Toward the Conversational User Interface, by Michael F. McTear , 2002, CL.

[48]  Milica Gasic,et al.  POMDP-Based Statistical Spoken Dialog Systems: A Review , 2013, Proceedings of the IEEE.

[49]  Mark Strassman,et al.  Case study: Development of the Find Friend Application , 2004 .

[50]  Jonathan Raper,et al.  Positioning techniques for location‐based services (LBS): characteristics and limitations of proposed solutions , 2001 .

[51]  Oliver Lemon,et al.  A Multithreaded Conversational Interface for Pedestrian Navigation and Question Answering , 2013, SIGDIAL Conference.

[52]  Oliver Lemon,et al.  Evaluating a city exploration dialogue system combining question-answering and pedestrian navigation , 2013, ACL 2013.