Prominence Driven Character Animation

This paper details the development of a fully automated system for character animation implemented in Autodesk Maya. The system uses prioritised speech events to algorithmically generate head, body, arms and leg movements alongside eyeblinks, eyebrow movements and lip-synching. In addition, gaze tracking is also generated automatically relative to the definition of focus objects- contextually important objects in the character’s worldview. The plugin uses an animation profile to store the relevant controllers and movements for a specific character, allowing any character to run with the system. Once a profile has been created, an audio file can be loaded and animated with a single button click. The average time to animate is between 2-3 minutes for 1 minute of speech, and the plugin can be used either as a first pass system for high quality work or as part of a batch animation workflow for larger amounts of content as exemplified in television and online dissemination channels.

[1]  Paul Boersma,et al.  Should jitter be measured by peak picking or by waveform matching , 2009 .

[2]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[3]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[4]  Roddy Cowie,et al.  Describing the emotional states that are expressed in speech , 2003, Speech Commun..

[5]  DeLiang Wang,et al.  Auditory Segmentation Based on Onset and Offset Analysis , 2007, IEEE Transactions on Audio, Speech, and Language Processing.

[6]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents: Research Articles , 2004 .

[7]  Marc Schröder,et al.  The German Text-to-Speech Synthesis System MARY: A Tool for Research, Development and Teaching , 2003, Int. J. Speech Technol..

[8]  Jehee Lee,et al.  Expressive Facial Gestures From Motion Capture Data , 2008, Comput. Graph. Forum.

[9]  J. Sundberg,et al.  Perceptual and acoustic correlates of abnormal voice qualities. , 1980, Acta oto-laryngologica.

[10]  Richard J. Davidson,et al.  Eye-blink rate predicts individual differences in pseudoneglect , 2010, Neuropsychologia.

[11]  M. Doughty,et al.  Further Assessment of Gender- and Blink Pattern-Related Differences in the Spontaneous Eyeblink Activity in Primary Gaze in Young Adult Humans , 2002, Optometry and vision science : official publication of the American Academy of Optometry.

[12]  Iain R. Murray,et al.  Toward the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion. , 1993, The Journal of the Acoustical Society of America.

[13]  André Schulz,et al.  Lateralization effects on the cardiac modulation of acoustic startle eye blink , 2009, Biological Psychology.

[14]  M. Doughty.  Consideration of Three Types of Spontaneous Eyeblink Activity in Normal Humans: during Reading and Video Display Terminal Use, in Primary Gaze, and while in Conversation , 2001, Optometry and vision science : official publication of the American Academy of Optometry.

[15]  M. Aramideh,et al.  Eyelid movements: behavioral studies of blinking in humans under different stimulus conditions. , 2003, Journal of neurophysiology.

[16]  Richard Williams,et al.  The Animator's Survival Kit , 2001 .

[17]  Bernhard Hommel,et al.  Reduced Spontaneous Eye Blink Rates in Recreational Cocaine Users: Evidence for Dopaminergic Hypoactivity , 2008, PloS one.

[18]  J. Pickett,et al.  The Acoustics of Speech Communication: Fundamentals, Speech Perception Theory, and Technology , 1998 .

[19]  Uri Hadar,et al.  Kinematics of head movements accompanying speech during conversation , 1983 .

[20]  Christoph Bregler,et al.  Mood swings: expressive speech animation , 2005, TOGS.

[21]  Rudolf Arnheim,et al.  Hand and Mind: What Gestures Reveal About Thought by David McNeill (review) , 2017 .

[22]  Cynthia Breazeal,et al.  Human social response toward humanoid robot's head and facial features , 2010, CHI Extended Abstracts.

[23]  Sergey Levine,et al.  Real-time prosody-driven synthesis of body language , 2009, ACM Trans. Graph..

[24]  Hans-Peter Seidel,et al.  Annotated New Text Engine Animation Animation Lexicon Animation Gesture Profiles MR : . . . JL : . . . Gesture Generation Video Annotated Gesture Script , 2007 .

[25]  M Collins,et al.  Blinking and corneal sensitivity , 1989, Acta ophthalmologica.

[26]  S. Drucker,et al.  The Role of Eye Gaze in Avatar Mediated Conversational Interfaces , 2000 .

[27]  C. Goodwin.  Action and embodiment within situated human interaction , 2000 .

[28]  Charlie Cullen,et al.  Reusable, Interactive, Multilingual Online Avatars , 2009, 2009 Conference for Visual Media Production.

[29]  E. Ponder,et al.  ON THE ACT OF BLINKING , 1927 .

[30]  Michael Neff,et al.  An annotation scheme for conversational gestures: how to economically capture timing and form , 2007, Lang. Resour. Evaluation.

[31]  Charlie Cullen,et al.  A VOWEL-STRESS EMOTIONAL SPEECH ANALYSIS METHOD , 2008 .

[32]  Norihiro Hagita,et al.  Messages embedded in gaze of interface agents --- impression management with agent's gaze , 2002, CHI.

[33]  Charlie Cullen,et al.  Emotional Speech Corpus Construction, Annotation and Distribution , 2008, LREC 2008.

[34]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[35]  Justine Cassell,et al.  Embodied conversational interface agents , 2000, CACM.

[36]  Charlie Cullen,et al.  CorpVis: An Online Emotional Speech Corpora Visualisation Interface , 2009, SAMT.