From Spatial Relations to Spatial Configurations

Spatial Reasoning from language is essential for natural language understanding. Supporting it requires a representation scheme that can capture spatial phenomena encountered in language as well as in images and videos.Existing spatial representations are not sufficient for describing spatial configurations used in complex tasks. This paper extends the capabilities of existing spatial representation languages and increases coverage of the semantic aspects that are needed to ground spatial meaning of natural language text in the world. Our spatial relation language is able to represent a large, comprehensive set of spatial concepts crucial for reasoning and is designed to support composition of static and dynamic spatial configurations. We integrate this language with the Abstract Meaning Representation (AMR) annotation schema and present a corpus annotated by this extended AMR. To exhibit the applicability of our representation scheme, we annotate text taken from diverse datasets and show how we extend the capabilities of existing spatial representation languages with fine-grained decomposition of semantics and blend it seamlessly with AMRs of sentences and discourse representations as a whole.

[1]  James Pustejovsky,et al.  The Revision of ISO-Space,Focused on the Movement Link , 2018, ACL 2018.

[2]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[3]  Daniel Gildea,et al.  The Necessity of Parsing for Predicate Argument Recognition , 2002, ACL.

[4]  Marie-Francine Moens,et al.  SemEval-2012 Task 3: Spatial Role Labeling , 2012, SemEval@NAACL-HLT.

[5]  Jordan Zlatev,et al.  Holistic spatial semantics of Thai , 2003 .

[6]  Julia Hockenmaier,et al.  Collaborative Dialogue in Minecraft , 2019, ACL.

[7]  J. Hois,et al.  Counterparts in Language and SpaceSimilarity and S-Connection , 2008, FOIS 2008.

[8]  Werner Kuhn,et al.  A Model of Spatial Reference Frames in Language , 2011, COSIT.

[9]  Jonathan Berant,et al.  Weakly Supervised Semantic Parsing with Abstract Examples , 2017, ACL.

[10]  Daniel Marcu,et al.  Learning Interpretable Spatial Operations in a Rich 3D Blocks World , 2017, AAAI.

[11]  Robin Cooper,et al.  Interfacing language, spatial perception and cognition in Type Theory with Records , 2017, J. Lang. Model..

[12]  Inderjeet Mani,et al.  SpatialML: Annotation Scheme, Corpora, and Tools , 2008, LREC.

[13]  Yoav Artzi,et al.  A Corpus of Natural Language for Visual Reasoning , 2017, ACL.

[14]  Christian Freksa,et al.  Qualitative Spatial Representation and Reasoning in the SparQ-Toolbox , 2006, Spatial Cognition.

[15]  Marie-Francine Moens,et al.  Spatial Role Labeling: Task Definition and Annotation Scheme , 2010, LREC.

[16]  Thora Tenbrink,et al.  A linguistic ontology of space for natural language processing , 2010, Artif. Intell..

[17]  Trevor Darrell,et al.  Learning to Reason: End-to-End Module Networks for Visual Question Answering , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Martha Palmer,et al.  Spatial AMR: Expanded Spatial Annotation in the Context of a Grounded Minecraft Corpus , 2020, LREC.

[19]  Marie-Francine Moens,et al.  Spatial Role Labeling Annotation Scheme , 2017 .

[20]  James Pustejovsky,et al.  SemEval-2015 Task 8: SpaceEval , 2015, *SEMEVAL.

[21]  Anthony G. Cohn,et al.  A Spatial Logic based on Regions and Connection , 1992, KR.

[22]  Christopher D. Manning,et al.  GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Kevin Duh,et al.  AMR Parsing as Sequence-to-Graph Transduction , 2019, ACL.

[24]  Philipp Koehn,et al.  Abstract Meaning Representation for Sembanking , 2013, LAW@ACL.

[25]  James Pustejovsky,et al.  Combining Deep Learning and Qualitative Spatial Reasoning to Learn Complex Structures from Sparse Examples with Noise , 2018, AAAI.