Generating animation from natural language texts and semantic analysis for motion search and scheduling

This paper presents an animation system that generates an animation from natural language texts such as movie scripts or stories. It also proposes a framework for a motion database that stores numerous motion clips for various characters. We have developed semantic analysis methods to extract information for motion search and scheduling from script-like input texts. Given an input text, the system searches for an appropriate motion clip in the database for each verb in the input text. Temporal constraints between verbs are also extracted from the input text and are used to schedule the motion clips found. In addition, when necessary, certain automatic motions such as locomotion, taking an instrument, changing posture, and cooperative motions are searched for in the database. An animation is then generated using an external motion synthesis system. With our system, users can make use of existing motion clips. Moreover, because it takes natural language text as input, even novice users can use our system.

[1]  Ruqian Lu,et al.  Automatic Generation of Computer Animation , 2001, Lecture Notes in Computer Science.

[2]  Michael F. Cohen,et al.  Verbs and Adverbs: Multidimensional Motion Interpolation , 1998, IEEE Computer Graphics and Applications.

[3]  Ken Perlin,et al.  Improv: a system for scripting interactive actors in virtual worlds , 1996, SIGGRAPH.

[4]  Norman I. Badler,et al.  A Parameterized Action Representation for Virtual Human Agents , 1998 .

[5]  Tsukasa Noma,et al.  Visualization of Temporal and Spatial Information in Natural Language Descriptions , 1996 .

[6]  Kaoru Sumi,et al.  Animated storytelling system via text , 2006, ACE '06.

[7]  Matthew J. Conway,et al.  Alice: Easy-to-Learn 3D Scripting for Novices , 1998 .

[8]  Norman I. Badler,et al.  Dynamically altering agent behaviors using natural language instructions , 2000, AGENTS '00.

[9]  Ruqian Lu,et al.  Automatic generation of computeranimation: using AI for movie animation , 2002 .

[10]  Bo Gyeong Kang,et al.  CAMEO - camera, audio and motion with emotion orchestration for immersive cinematography , 2008, ACE '08.

[11]  Sergey Levine,et al.  Real-time prosody-driven synthesis of body language , 2009, SIGGRAPH 2009.

[12]  Michael Gleicher,et al.  Automated extraction and parameterization of motions in large data sets , 2004, SIGGRAPH 2004.

[13]  Dan Klein,et al.  Fast Exact Inference with a Factored Model for Natural Language Parsing , 2002, NIPS.

[14]  Masaki Oshita.  Smart motion synthesis , 2007, SIGGRAPH '07.

[15]  上田 博唯,et al.  TVML(TV program Making Language) : Automatic TV Program Generation from Text-based Script , 1999 .

[16]  Richard Sproat,et al.  WordsEye: an automatic text-to-scene conversion system , 2001, SIGGRAPH.

[17]  Masaki Oshita.  Generating Animation from Natural Language Texts and Framework of Motion Database , 2009, 2009 International Conference on CyberWorlds.

[18]  Hozumi Tanaka,et al.  K2: Animated Agents that Understand Speech Commands and Perform Actions , 2004, PRICAI.

[19]  Annette Lessmöllmann.  Automatic Generation of Computer Animation: Using AI for Movie Animation , 2002, Künstliche Intell..

[20]  Sung Yong Shin,et al.  On-line locomotion generation based on motion blending , 2002, SCA '02.

[21]  Terry Winograd,et al.  Understanding natural language , 1974 .

[22]  Charles J. Fillmore,et al.  THE CASE FOR CASE. , 1967 .