From Descriptions to Depictions: A Conceptual Framework

People use verbal descriptions and graphical depictions to communicate spatial information, thus externalizing their spatial mental representations. In many situations, such as in emergency response, the ability to translate the content of verbal descriptions into a sketch map could greatly assist with the interpretation of the message. In this paper, we present an outline of a semi-automatic framework enabling seamless transition between verbal descriptions and graphical sketches of precinct-scale urban environments. The proposed framework relies on a three-step approach: NL parsing, with spatial named entity and spatial relation recognition in natural language text; the construction of a spatial Property Graph capturing the spatial relationships between pairs of entities; and the sketch drawing step where the identified entities are dynamically placed on a canvas in a manner that minimizes conflicts between the verbalized spatial relationships, thus providing a plausible representation of the described environment. The approach is manually demonstrated on a natural language description of a university campus, and the opportunities and challenges of the suggested framework are discussed. The paper concludes by highlighting the contributions of the framework and by providing insights for its actual implementation.

[1]  Kenneth C. Litkowski,et al.  SemEval-2007 Task 06: Word-Sense Disambiguation of Prepositions , 2007, Fourth International Workshop on Semantic Evaluations (SemEval-2007).

[2]  Julia Hirschberg,et al.  Spatial Relations in Text-to-Scene Conversion , 2010 .

[3]  Julia Hirschberg,et al.  Frame Semantics in Text-to-Scene Generation , 2010, KES.

[4]  Siobhan Chapman Logic and Conversation , 2005 .

[5]  Tiejun Zhao,et al.  The Extraction of Trajectories from Real Texts Based on Linear Classification , 2007, NODALIDA.

[6]  Luc De Raedt,et al.  Relational Learning for Spatial Relation Extraction from Natural Language , 2011, ILP.

[7]  Dan Tappan,et al.  Knowledge-Based Spatial Reasoning for Scene Generation from Text Descriptions , 2008, AAAI.

[8]  R. Downs,et al.  Cognitive Maps and Spatial Behaviour: Process and Products , 2011 .

[9]  Jochen L. Leidner Toponym resolution in text: annotation, evaluation and applications of spatial grounding , 2007, SIGF.

[10]  Christiane Fellbaum,et al.  Book Reviews: WordNet: An Electronic Lexical Database , 1999, CL.

[11]  John A. Bateman,et al.  Language and Space: a two-level semantic approach based on principles of ontological engineering , 2010, Int. J. Speech Technol..

[12]  M. Denis The description of routes : A cognitive approach to the production of spatial discourse , 1997 .

[13]  J. Jonides,et al.  Evidence of hierarchies in cognitive maps , 1985, Memory & cognition.

[14]  Christian Freksa,et al.  Using Orientation Information for Qualitative Spatial Reasoning , 1992, Spatio-Temporal Reasoning.

[15]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[16]  A. Schwering,et al.  The Accuracy of Sketched Spatial Relations : How Cognitive Errors Affect Sketch Representation , 2009 .

[17]  Anthony G. Cohn,et al.  A Spatial Logic based on Regions and Connection , 1992, KR.

[18]  Max J. Egenhofer,et al.  Query Processing in Spatial-Query-by-Sketch , 1997, J. Vis. Lang. Comput..

[19]  Roger T. Hartley,et al.  Knowledge-based spatial reasoning for automated scene generation from text descriptions , 2004 .

[20]  Chiara Meneghetti,et al.  The Mental Representation Derived from Spatial Descriptions is North-Up Oriented: The Role of Visuo-spatial Abilities , 2012, Spatial Cognition.

[21]  Kevin Lynch,et al.  The Image of the City , 1960 .

[22]  Daniel R. Montello,et al.  Spatial Information Theory A Theoretical Basis for GIS , 1995, Lecture Notes in Computer Science.

[23]  Barbara Tversky,et al.  What do Sketches Say about Thinking , 2002 .

[24]  P. Johnson-Laird,et al.  Conditionals: A Theory of Meaning, Pragmatics, and Inference , 2002 .

[25]  Michel Mainguenaud Modelling the Network Component of Geographical Information Systems , 1995, Int. J. Geogr. Inf. Sci..

[26]  Max J. Egenhofer,et al.  A visual tool for querying geographic databases , 2000, AVI '00.

[27]  Marie-Francine Moens,et al.  From language towards formal spatial calculi , 2010 .

[28]  T. McNamara,et al.  Subjective hierarchies in spatial memory. , 1989, Journal of experimental psychology. Learning, memory, and cognition.

[29]  Matthew Crosby,et al.  Association for the Advancement of Artificial Intelligence , 2014 .

[30]  Tobias Meilinger,et al.  The Network of Reference Frames Theory: A Synthesis of Graphs and Cognitive Maps , 2008, Spatial Cognition.

[31]  Daniela Richter,et al.  Zooming In-Zooming Out Hierarchies in Place Descriptions , 2013, Progress in Location-Based Services.

[32]  RalfHiutmut Gtiting,et al.  GraphDB : Modeling and Querying Graphs in Databases , 1998 .

[33]  Andreas D. Blaser User Interaction in a Sketch-Based GIS User Interface , 1997, COSIT.

[34]  Marie-Francine Moens,et al.  Spatial role labeling: Towards extraction of spatial relations from natural language , 2011, TSLP.

[35]  Paul U. Lee,et al.  Pictorial and Verbal Tools for Conveying Routes , 1999, COSIT.

[36]  Gabriel M. Kuper,et al.  A new approach to database logic , 1984, PODS.

[37]  Daniel Hernández,et al.  Relative Representation of Spatial Knowledge: The 2-D Case , 1991 .

[38]  Lidia Fraczak Generating “mental maps” from route descriptions , 1998 .

[39]  Jan Oliver Wallgrün,et al.  Qualitative Spatial Reasoning for Topological Map Learning , 2010, Spatial Cogn. Comput..

[40]  John A. Bateman,et al.  The Role of Conceptual and Linguistic Ontologies in Interpreting Spatial Discourse , 2007 .

[41]  Hubert Cuyckens,et al.  The Oxford handbook of cognitive linguistics , 2010 .

[42]  Thora Tenbrink,et al.  A linguistic ontology of space for natural language processing , 2010, Artif. Intell..

[43]  Patric Jensfelt,et al.  Functional topological relations for qualitative spatial representation , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[44]  J. Zlatev Spatial Semantics , 2006 .

[45]  Fritz Wysotzki,et al.  Inference and Visualization of Spatial Relations , 2000, Spatial Cognition.

[46]  Barbara Tversky,et al.  Cognitive Maps, Cognitive Collages, and Spatial Mental Models , 1993, COSIT.

[47]  Jan Hidders,et al.  Typing Graph-Manipulation Operations , 2003, ICDT.

[48]  Andrew U. Frank,et al.  Qualitative Spatial Reasoning: Cardinal Directions as an Example , 1996, Int. J. Geogr. Inf. Sci..

[49]  Ajay Mehra The Development of Social Network Analysis: A Study in the Sociology of Science , 2005 .

[50]  Kai-Florian Richter,et al.  Before or After: Prepositions in Spatially Constrained Systems , 2006, Spatial Cognition.

[51]  M. Egenhofer Categorizing Binary Topological Relations Between Regions, Lines, and Points in Geographic Databases , 1998 .

[52]  Barbara Tversky,et al.  Visualizing Thought , 2011, Top. Cogn. Sci..

[53]  L. Barsalou Grounded cognition. , 2008, Annual review of psychology.

[54]  Andrew U. Frank,et al.  Theories and Methods of Spatio-Temporal Reasoning in Geographic Space , 1992, Lecture Notes in Computer Science.

[55]  Tad T. Brunyé,et al.  Moving through imagined space: Mentally simulating locomotion during spatial description reading. , 2010, Acta psychologica.

[56]  John D. Kelleher A perceptually based computational framework for the interpretation of spatial language , 2003 .

[57]  Marko A. Rodriguez,et al.  Constructions from Dots and Lines , 2010, ArXiv.

[58]  Chris Stolte,et al.  Rendering effective route maps: improving usability through generalization , 2001, SIGGRAPH.