VoxSim: A Visual Platform for Modeling Motion Language

Much existing work in text-to-scene generation focuses on generating static scenes. By introducing a focus on motion verbs, we integrate dynamic semantics into a rich formal model of events to generate animations in real time that correlate with human conceptions of the event described. This paper presents a working system that generates these animated scenes over a test set, discussing challenges encountered and describing the solutions implemented.

[1]  Lars Kulik.  Qualitative Spatial Change , 2002, Künstliche Intell..

[2]  James Pustejovsky,et al.  VoxML: A Visualization Modeling Language , 2016, LREC.

[3]  Karin Schwab,et al.  Unity Game Development Essentials , 2016 .

[4]  James Pustejovsky,et al.  The Qualitative Spatial Dynamics of Motion in Language , 2011, Spatial Cogn. Comput..

[5]  R. Shaw,et al.  Perceiving, Acting and Knowing : Toward an Ecological Psychology , 1978 .

[6]  B. Bergen.  Louder Than Words: The New Science of How the Mind Makes Meaning , 2012 .

[7]  James Pustejovsky,et al.  The Generative Lexicon , 1995, CL.

[8]  James J. Gibson,et al.  The Ecological Approach to Visual Perception: Classic Edition , 2014 .

[9]  Anthony G. Cohn,et al.  A Spatial Logic based on Regions and Connection , 1992, KR.

[10]  Richard Sproat,et al.  WordsEye: an automatic text-to-scene conversion system , 2001, SIGGRAPH.

[11]  Slav Petrov,et al.  Globally Normalized Transition-Based Neural Networks , 2016, ACL.

[12]  Andrew McCallum,et al.  Transition-based Dependency Parsing with Selectional Branching , 2013, ACL.

[13]  Richard Johansson,et al.  Automatic Text-to-Scene Conversion in the Traffic Accident Domain , 2005, IJCAI.

[14]  Chaman L. Sabharwal,et al.  RCC-3D: Qualitative Spatial Reasoning in 3D , 2010, CAINE.

[15]  James F. Allen,et al.  TRIPS: An Integrated Intelligent Problem-Solving Assistant , 1998, AAAI/IAAI.

[16]  三嶋 博之.  The theory of affordances , 2008 .

[17]  James Pustejovsky,et al.  On the Representation of Inferences and their Lexicalization , 2013 .

[18]  Christopher Potts,et al.  Text to 3D Scene Generation with Rich Lexical Grounding , 2015, ACL.

[19]  J. Pustejovsky.  Dynamic Event Structure and Habitat Theory , 2013 .

[20]  Mehul Bhatt,et al.  Modelling Dynamic Spatial Systems in the Situation Calculus , 2008, Spatial Cogn. Comput..

[21]  James Pustejovsky,et al.  Generating Simulations of Motion Events from Verbal Descriptions , 2014, *SEMEVAL.

[22]  Max J. Egenhofer,et al.  The 9+-Intersection for Topological Relations between a Directed Line Segment and a Region , 2007, BMI.

[23]  James Pustejovsky,et al.  Multimodal Semantic Simulations of Linguistically Underspecified Motion Events , 2016, Spatial Cognition.