Dynamically altering agent behaviors using natural language instructions

Smart avatars are virtual human representations controlled by real people. Given instructions interactively, smart avatars can act as autonomous or reactive agents. During a real-time simulation, a user should be able to dynamically refine his or her avatar’s behavior in reaction to simulated stimuli without having to undertake a lengthy off-line programming session. In this paper, we introduce an architecture, which allows users to input immediate or persistent instructions using natural language and see the agents’ resulting behavioral changes in the graphical output of the simulation.

[1]  References , 1971 .

[2]  李幼升,et al.  Ph , 1989 .

[3]  Rodney A. Brooks,et al.  A robot that walks; emergent behaviors from a carefully evolved network , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[4]  Stuart M. Shieber,et al.  Synchronous Tree-Adjoining Grammars , 1990, COLING.

[5]  Anne Abeillé,et al.  A Lexicalized Tree Adjoining Grammar for English , 1990 .

[6]  David Chapman,et al.  Vision, instruction, and action , 1990 .

[7]  Timothy W. Bickmore,et al.  A basic agent , 1990, Comput. Intell..

[8]  Bonnie Webber,et al.  Animation from instructions , 1991 .

[9]  David Zeltzer,et al.  Task-level graphical simulation: abstraction, representation, and control , 1991 .

[10]  Daniel Thalmann,et al.  Simulation of Facial Muscle Actions Based on Rational Free Form Deformations , 1992, Comput. Graph. Forum.

[11]  Bonnie Webber,et al.  Plan recognition in understanding instructions , 1992 .

[12]  Aravind K. Joshi,et al.  Tree-adjoining grammars and lexicalized grammars , 1992, Tree Automata and Languages.

[13]  R. James Firby.  Task Networks for Controlling Continuous Processes , 1994, AIPS.

[14]  S. Shieber.  RESTRICTING THE WEAK‐GENERATIVE CAPACITY OF SYNCHRONOUS TREE‐ADJOINING GRAMMARS , 1994, Comput. Intell..

[15]  Norman I. Badler,et al.  Behavioral control for real-time simulated human agents , 1995, I3D '95.

[16]  John E. Laird,et al.  Flexibly Instructable Agents , 1995, J. Artif. Intell. Res..

[17]  Norman I. Badler,et al.  Instructions, Intentions and Expectations , 1995, Artif. Intell..

[18]  Mark Lutz,et al.  Programming Python , 1996 .

[19]  Norman I. Badler,et al.  Task-Level Object Grasping for Simulated Agents , 1996, Presence: Teleoperators & Virtual Environments.

[20]  Norman I. Badler,et al.  Decision networks for integrating the behaviors of virtual agents and avatars , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[21]  Ken Perlin,et al.  Improv: a system for scripting interactive actors in virtual worlds , 1996, SIGGRAPH.

[22]  Martha Palmer,et al.  Final Report to Air Force HRGA Regarding Feasibility of Natural Language Text Generation from Task Networks for Use in Automatic Generation of Technical Order from DEPTH Simulations , 1997 .

[23]  S. Narayanan.  Talking the Talk is Like Walking the Walk : A Computational Model of Verbal Aspect , 1997 .

[24]  W. Lewis Johnson,et al.  Steve: an animated pedagogical agent for procedural training in virtual environments , 1997, SGAR.

[25]  Computer Animation for Virtual Humans , 1998, IEEE Computer Graphics and Applications.

[26]  XTAG Research Group,et al.  A Lexicalized Tree Adjoining Grammar for English , 1998, ArXiv.

[27]  Sue Ki Wilcox.  Web Developer.com Guide to 3D Avatars , 1998 .

[28]  Juliet C. Bourne.  Generating Adequate Instructions: Knowing When to Stop , 1998, AAAI/IAAI.

[29]  N. Magnenat-Thalmann,et al.  Motion Control of Virtual Humans , 1998, IEEE Computer Graphics and Applications.

[30]  William Schuler.  Preserving Semantic Dependencies in Synchronous Tree Adjoining Grammar , 1999, ACL.

[31]  W. Lewis Johnson,et al.  Animated Agents for Procedural Training in Virtual Reality: Perception, Cognition, and Motor Control , 1999, Appl. Artif. Intell..

[32]  Daniel Thalmann,et al.  A behavioral interface to simulate agent-object interactions in real time , 1999, Proceedings Computer Animation 1999.

[33]  Norman I. Badler,et al.  Virtual human animation based on movement observation and cognitive behavior models , 1999, Proceedings Computer Animation 1999.

[34]  Norman I. Badler,et al.  Animation control for real-time virtual humans , 1999, CACM.

[35]  Norman I. Badler,et al.  Smart avatars in JackMOO , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).