A Context-Dependent Model of Proximity in Physically Situated Environments

The paper presents a computational model for a context-dependent analysis of a physical environment in terms of spatial proximity. The model provides a basis for grounding linguistic analyses of spatial expressions in visual perception. The model uses potential fields to model spatial proximity. It has been implemented, and when combined with a handcrafted grammar, is used to enable a conversational robot to carry out a situated dialogue with a human. The key concept in our approach is defining the region that is proximal to a landmark based on the spatial configuration of other objects in the scene. The model extends existing approaches to proximity by including object salience (visual, discourse) and interference effects between multiple objects that could act as landmarks. Theoretically, the model can help motivate the choice between topological and projective prepositions, and provides a basis for defining regions with vague spatial extent.

[1]  D. Fara Shifting sands: An interest relative theory of vagueness , 2000 .

[2]  Christopher Kennedy,et al.  Towards a Grammar of Vagueness∗ , 2022 .

[3]  E. Krahmer,et al.  Efficient Generation of Descriptions in Context , 1999 .

[4]  J. Peregrin LINGUISTICS AND PHILOSOPHY , 1998 .

[5]  Christopher Kennedy,et al.  Scale Structure, Degree Modification, and the Semantics of Gradable Predicates , 2005 .

[6]  A. Kyburg,et al.  Fitting Words: Vague Language in Context , 2000 .

[7]  Josef van Genabith,et al.  A Computational Model of the Referential Semantics of Projective Prepositions , 2006 .

[8]  Deb Roy,et al.  Grounded Semantic Composition for Visual Scenes , 2011, J. Artif. Intell. Res..

[9]  Gordon D. Logan,et al.  A computational analysis of the apprehension of spatial relations , 1996 .

[10]  Josef van Genabith,et al.  Visual Salience and Reference Resolution in Simulated 3-D Environments , 2004, Artificial Intelligence Review.

[11]  Jonathan Ginzburg,et al.  Proceedings of COLING 2004 , 2004 .

[12]  William Schuler,et al.  Computational Properties of Environment-based Disambiguation , 2001, ACL.

[13]  G. Logan Linguistic and Conceptual Control of Visual Spatial Attention , 1995, Cognitive Psychology.

[14]  Laura A. Carlson,et al.  Grounding spatial language in perception: an empirical and computational investigation. , 2001, Journal of experimental psychology. General.

[15]  Drew H. Abney,et al.  Journal of Experimental Psychology : Human Perception and Performance Influence of Musical Groove on Postural Sway , 2015 .

[16]  Amitabha Mukerjee,et al.  Conceptual description of visual scenes from linguistic models , 2000, Image Vis. Comput..

[17]  Paul R. Cohen,et al.  Toward natural language interfaces for robotic agents: grounding linguistic meaning in sensors , 2000, AGENTS '00.

[18]  Klaus-Peter Gapp Basic Meanings of Spatial Relations: Computation and Evaluation in 3D Space , 1994, AAAI.

[19]  Roger K. Moore Computer Speech and Language , 1986 .

[20]  Robert Dale,et al.  Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions , 1995, Cogn. Sci..

[21]  G. Logan Spatial attention and the apprehension of spatial relations. , 1994, Journal of experimental psychology. Human perception and performance.

[22]  Thomas Rist,et al.  Natural Language Access to Visual Data: Dealing with Space and Movement , 1989 .

[23]  Emiel Krahmer,et al.  The influence of target size and distance on the production of speech and gesture in multimodal referring expressions , 2004, INTERSPEECH.

[24]  Jason Baldridge,et al.  Coupling CCG and Hybrid Logic Dependency Semantics , 2002, ACL.

[25]  Jason Baldridge,et al.  Multi-Modal Combinatory Categorial Grammar , 2003, EACL.

[26]  Gerhard Sagerer,et al.  A three-dimensional spatial model for the interpretation of image data , 1998, IJCAI 1995.

[27]  Patrick Olivier,et al.  Quantitative perceptual representation of prepositional semantics , 2004, Artificial Intelligence Review.

[28]  Michael White,et al.  Efficient Realization of Coordinate Structures in Combinatory Categorial Grammar , 2006 .

[29]  Philip R. Cohen,et al.  Referring as a Collaborative Process , 2003 .