Reference Resolution for Pedestrian Wayfinding Systems

References to objects in our physical environment are common especially in language about wayfinding. Advanced wayfinding systems that interact with the pedestrian by means of (spoken) natural language therefore need to be able to resolve references given by pedestrians (i.e. understand what entity the pedestrian is referring to). The contribution of this paper is a probabilistic approach to reference resolution in a large-scale, real city environment, where the context changes constantly as the pedestrians are moving. The geographic situation, including information about objects’ location and type, is represented using OpenStreetMap data.

[1]  D. Roy.  Grounding words in perception and action: computational insights , 2005, Trends in Cognitive Sciences.

[2]  Alexander Koller,et al.  The Potsdam NLG systems at the GIVE-2.5 Challenge , 2011, ENLG.

[3]  Raymond J. Mooney,et al.  Learning to Connect Language and Perception , 2008, AAAI.

[4]  Kristinn R. Thórisson,et al.  Simulated Perceptual Grouping: An Application to Human-Computer Interaction , 2019, Proceedings of the Sixteenth Annual Conference of the Cognitive Science Society.

[5]  Mario Fritz,et al.  A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input , 2014, NIPS.

[6]  Emiel Krahmer,et al.  Improving Route Directions: The Role of Intersection Type and Visual Clutter for Spatial Reference , 2015 .

[7]  Kees van Deemter,et al.  Reference and the facilitation of search in spatial domains , 2014 .

[8]  Johan Boye,et al.  SpaceRef: A corpus of street-level geographic descriptions , 2016, LREC.

[9]  Takenobu Tokunaga,et al.  A Unified Probabilistic Approach to Referring Expressions , 2012, SIGDIAL Conference.

[10]  Patrick Weber,et al.  OpenStreetMap: User-Generated Street Maps , 2008, IEEE Pervasive Computing.

[11]  Johan Boye,et al.  Resolving Spatial References using Crowdsourced Geographical Data , 2015, NODALIDA.

[12]  John D. Kelleher,et al.  Visual Salience and Reference Resolution in Situated Dialogues: A Corpus-based Evaluation , 2010, AAAI Fall Symposium: Dialog with Robots.

[13]  Joakim Gustafson,et al.  Walk This Way: Spatial Grounding for City Exploration , 2014, Natural Interaction with Robots, Knowbots and Smartphones, Putting Spoken Dialog Systems into Practice.

[14]  Marko Modsching,et al.  Field trial on GPS Accuracy in a medium size city: The influence of built- up 1 , 2006 .

[15]  Rakesh Gupta,et al.  Situated Language Understanding at 25 Miles per Hour , 2014, SIGDIAL Conference.

[16]  Luke S. Zettlemoyer,et al.  Learning from Unscripted Deictic Gesture and Language for Human-Robot Interactions , 2014, AAAI.

[17]  Nate Blaylock.  Semantic Annotation of Street-Level Geospatial Entities , 2011, 2011 IEEE Fifth International Conference on Semantic Computing.

[18]  M. Denis.  The description of routes : A cognitive approach to the production of spatial discourse , 1997 .

[19]  Laurent Romary,et al.  Reference Resolution within the Framework of Cognitive Grammar , 2009, ArXiv.

[20]  John D. Kelleher,et al.  Information Fusion for Visual Reference Resolution in Dynamic Situated Dialogue , 2006, PIT.

[21]  Takenobu Tokunaga,et al.  Multi-modal Reference Resolution in Situated Dialogue by Integrating Linguistic and Extra-Linguistic Clues , 2011, IJCNLP.

[22]  David Schlangen,et al.  Simple Learning and Compositional Application of Perceptually Grounded Word Meanings for Incremental Reference Resolution , 2015, ACL.

[23]  Stephan Winter,et al.  Enriching Wayfinding Instructions with Local Landmarks , 2002, GIScience.

[24]  Jayant Krishnamurthy,et al.  Jointly Learning to Parse and Perceive: Connecting Natural Language to the Physical World , 2013, TACL.

[25]  Deb Roy,et al.  Probabilistic grounding of situated speech using plan recognition and reference resolution , 2005, ICMI '05.

[26]  Johan Boye,et al.  Learning landmark salience models from users’ route instructions , 2016, J. Locat. Based Serv..

[27]  Benjamin Kuipers,et al.  Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions , 2006, AAAI.

[28]  Michela Bertolotto,et al.  Geographic knowledge extraction and semantic similarity in OpenStreetMap , 2013, Knowledge and Information Systems.

[29]  Luke S. Zettlemoyer,et al.  A Joint Model of Language and Perception for Grounded Attribute Learning , 2012, ICML.

[30]  Elia Bruni,et al.  Multimodal Distributional Semantics , 2014, J. Artif. Intell. Res..