Implementing a non-modular theory of language production in an embodied

This chapter discusses and assesses the feasibility of operationalising Growth Point Theory's model of language production in embodied conversational agents (ECAs). First, the chapter outlines the cornerstones of non-modular Growth Point Theory and its empirical basis. It then gives an overview of gesture and speech production models that are currently realised in ECAs, and discuss their potential and limitations with respect to which characteristics of natural speech and gesture they can account for. Finally, it discusses which requirements a technical model must meet in order to be more compatible with Growth Point Theory.

[1]  Betty Tuller,et al.  A “Dynamic Pattern” Perspective on the Control and Coordination of Movement , 1983 .

[2]  Stefan Kopp,et al.  Verbal or Visual? How Information is Distributed across Speech and Gesture in Spatial Dialog , 2006 .

[3]  J. Kelso,et al.  Skilled actions: a task-dynamic approach. , 1987, Psychological review.

[4]  Justine Cassell,et al.  BEAT: the Behavior Expression Animation Toolkit , 2001, Life-like characters.

[5]  P. Radford,et al.  Imitation , 2023 .

[6]  Ipke Wachsmuth,et al.  A model for the representation and processing of shape in coverbal iconic gestures , 2005 .

[7]  Cornelia Müller,et al.  Redebegleitende Gesten : Kulturgeschichte, Theorie, Sprachvergleich , 1998 .

[8]  Susan Duncan,et al.  Growth points in thinking-for-speaking , 1998 .

[9]  D. Laplane.  Thought and language. , 1992, Behavioural neurology.

[10]  Stefan Kopp,et al.  Synthesizing multimodal utterances for conversational agents , 2004, Comput. Animat. Virtual Worlds.

[11]  A. Kendon.  Gesticulation and Speech: Two Aspects of the Process of Utterance , 1981 .

[12]  D. McNeill.  Gesture and Thought , 2005 .

[13]  Stefan Kopp,et al.  Towards integrated microplanning of language and iconic gesture for multimodal output , 2004, ICMI '04.

[14]  Susan Duncan,et al.  Gesture, verb aspect, and the nature of iconic imagery in natural discourse , 2002 .

[15]  Timo Sowa.  Towards the integration of shape-related information in 3-D gestures and speech , 2006, ICMI '06.

[16]  David McNeill,et al.  Language and Gesture: Frontmatter , 2000 .

[17]  Stefan Kopp,et al.  Situated interaction with a virtual human - perception, action, and cognition , 2006 .

[18]  J. D. Ruiter.  The production of gesture and speech , 2000 .

[19]  M. Studdert-Kennedy.  Hand and Mind: What Gestures Reveal About Thought. , 1994 .

[20]  Paul Thagard,et al.  Dynamic Imagery: A Computational Model of Motion and Visual Analogy , 2002 .

[21]  Ying Choon Wu,et al.  Meaningful gestures: electrophysiological indices of iconic gesture comprehension. , 2005, Psychophysiology.

[22]  B. Butterworth,et al.  Iconic gestures, imagery, and word retrieval in speech , 1997 .

[23]  L. Vygotsky,et al.  Thought and Language , 1963 .

[24]  Karen Emmorey,et al.  Using space to describe space: Perspective in speech, sign, and gesture , 2000, Spatial Cogn. Comput..

[25]  Stefan Kopp,et al.  Imitation Learning and Response Facilitation in Embodied Agents , 2006, IVA.

[26]  A. Battersby.  Plans and the Structure of Behavior , 1968 .

[27]  Ehud Reiter,et al.  Book Reviews: Building Natural Language Generation Systems , 2000, CL.

[28]  N. J. Enfield,et al.  On linear segmentation and combinatorics in co-speech gesture: A symmetry-dominance construction in Lao fish trap descriptions* , 2004 .

[29]  Michael A. Arbib,et al.  The handbook of brain theory and neural networks , 1995, A Bradford book.

[30]  Janice I. Glasgow,et al.  THE IMAGERY DEBATE REVISITED: A COMPUTATIONAL PERSPECTIVE , 1993 .

[31]  S. Nobe.  Language and Gesture: Where do most spontaneous representational gestures actually occur with respect to speech? , 2000 .

[32]  S. Kelly,et al.  Neural correlates of bimodal speech and gesture comprehension , 2004, Brain and Language.

[33]  S. Kosslyn.  The medium and the message in mental imagery: A theory. , 1981 .

[34]  S. Kosslyn.  Image and mind , 1982 .

[35]  Justine Cassell,et al.  Human conversation as a system framework: designing embodied conversational agents , 2001 .

[36]  L. Barsalou,et al.  Whither structured representation? , 1999, Behavioral and Brain Sciences.

[37]  D. Slobin.  Thinking for Speaking , 1987 .