Generating Animations from Screenplays

Automatically generating animation from natural language text finds application in a number of areas e.g. movie script writing, instructional videos, and public safety. However, translating natural language text into animation is a challenging task. Existing text-to-animation systems can handle only very simple sentences, which limits their applications. In this paper, we develop a text-to-animation system which is capable of handling complex sentences. We achieve this by introducing a text simplification step into the process. Building on an existing animation generation system for screenwriting, we create a robust NLP pipeline to extract information from screenplays and map them to the system's knowledge base. We develop a set of linguistic transformation rules that simplify complex sentences. Information extracted from the simplified sentences is used to generate a rough storyboard and video depicting the text. Our sentence simplification module outperforms existing systems in terms of BLEU and SARI metrics.We further evaluated our system via a user study: 68 % participants believe that our system generates reasonable animation from input screenplays.

[1]  Won-Sook Lee,et al.  Visualizing Natural Language Descriptions , 2016, ACM Comput. Surv..

[2]  R. Likert “Technique for the Measurement of Attitudes, A” , 2022, The SAGE Encyclopedia of Research Design.

[3]  Daniel Ferrés,et al.  YATS: Yet Another Text Simplifier , 2016, NLDB.

[4]  Advaith Siddharthan,et al.  Text Simplification using Typed Dependencies: A Comparision of the Robustness of Different Generation Strategies , 2011, ENLG.

[5]  Mubbasir Kapadia,et al.  PICA: Proactive Intelligent Conversational Agent for Interactive Narratives , 2018, IVA.

[6]  Oren Etzioni,et al.  Open Language Learning for Information Extraction , 2012, EMNLP.

[7]  Markus H. Gross,et al.  CARDINAL: Computer Assisted Authoring of Movie Scripts , 2018, IUI.

[8]  Luke S. Zettlemoyer,et al.  Deep Contextualized Word Representations , 2018, NAACL.

[9]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[10]  Mark Johnson,et al.  An Improved Non-monotonic Transition System for Dependency Parsing , 2015, EMNLP.

[11]  Christopher D. Manning,et al.  Leveraging Linguistic Structure For Open Domain Information Extraction , 2015, ACL.

[12]  Luke S. Zettlemoyer,et al.  Deep Semantic Role Labeling: What Works and What’s Next , 2017, ACL.

[13]  Ivan Titov,et al.  Modeling Semantic Expectation: Using Script Knowledge for Referent Prediction , 2017, TACL.

[14]  Ashutosh Modi,et al.  Event Embeddings for Semantic Script Modeling , 2016, CoNLL.

[15]  Denilson Barbosa,et al.  Improving Open Relation Extraction via Sentence Re-Structuring , 2014, LREC.

[16]  Richard Johansson,et al.  Carsim: A system to visualize written road accident reports as animated 3D scenes , 2004 .

[17]  André Freitas,et al.  A Survey on Open Information Extraction , 2018, COLING.

[18]  Zhi-Qiang Liu,et al.  Script visualization (ScriptViz): a smart system that makes writing fun , 2006, Soft Comput..

[19]  Hong Yu,et al.  Sentence Simplification with Memory-Augmented Neural Networks , 2018, NAACL.

[20]  HassaniKaveh,et al.  Visualizing Natural Language Descriptions , 2016 .

[21]  Markus H. Gross,et al.  Computer-Assisted Authoring for Natural Language Story Scripts , 2018, AAAI.

[22]  Ashutosh Modi,et al.  Modeling common sense knowledge via scripts , 2017 .

[23]  Gonzalo Navarro,et al.  A guided tour to approximate string matching , 2001, CSUR.

[24]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[25]  Chris Callison-Burch,et al.  Optimizing Statistical Machine Translation for Text Simplification , 2016, TACL.

[26]  Ralph Grishman,et al.  Corpus-based Parsing and Sublanguage Studies , 1998 .

[27]  Ruqian Lu,et al.  Automatic generation of computeranimation: using AI for movie animation , 2002 .

[28]  Xiaojun Wan,et al.  Automatic Text Simplification , 2018, Computational Linguistics.

[29]  Joan Condell,et al.  SceneMaker: Intelligent Multimodal Visualisation of Natural Language Scripts , 2009, AICS.

[30]  Luciano Del Corro,et al.  ClausIE: clause-based open information extraction , 2013, WWW.

[31]  Timo Järvinen,et al.  A non-projective dependency parser , 1997, ANLP.

[32]  Sergiu Nisioi,et al.  Exploring Neural Text Simplification Models , 2017, ACL.

[33]  Ari Rappoport,et al.  BLEU is Not Suitable for the Evaluation of Text Simplification , 2018, EMNLP.

[34]  Ruslan Salakhutdinov,et al.  Generating Images from Captions with Attention , 2015, ICLR.

[35]  Diana Maynard,et al.  JAPE: a Java Annotation Patterns Engine , 2000 .

[36]  Minhua Ma,et al.  Virtual human animation in natural language visualisation , 2007, Artificial Intelligence Review.

[37]  Ivan Titov,et al.  Inducing Neural Models of Script Knowledge , 2014, CoNLL.

[38]  R. Michael Young,et al.  Automated Screenplay Annotation for Extracting Storytelling Knowledge , 2017, AIIDE Workshops.