Instructions, Intentions and Expectations

Abstract Based on an ongoing attempt to integrate Natural Language instructions with human figure animation, we demonstrate that agents' understanding and use of instructions can complement what they can derive from the environment in which they act. We focus on two attitudes that contribute to agents' behavior—their intentions and their expectations—and shown how Natural Language instructions contribute to such attitudes in ways that complement the environment. We also show that instructions can require more than one context of interpretation and thus that agents' understanding of instructions can evolve as their activity progresses. A significant consequence is that Natural Language understanding in the context of behavior cannot simply be treated as “front end” processing, but rather must be integrated more deeply into the processes that guide an agent's behavior and respond to its perceptions.

[1]  David Chapman,et al.  Vision, instruction, and action , 1990 .

[2]  Joseph Bates,et al.  Real-time Control of Animated Broad Agents , 1993 .

[3]  Martha E. Pollack,et al.  Inferring domain plans in question-answering , 1986 .

[4]  Mark T. Maybury,et al.  Intelligent multimedia interfaces , 1994, CHI Conference Companion.

[5]  Robin F. Karlin.  Defining the Semantics of Verbal Modifiers in the Domain of Cooking Tasks , 1988, ACL.

[6]  Austin Tate,et al.  Generating Project Networks , 1977, IJCAI.

[7]  David J. Israel,et al.  Plans and resource‐bounded practical reasoning , 1988, Comput. Intell..

[8]  Donia Scott,et al.  Knowledge, Intention, Rhetoric: Levels of Variation in Multilingual Instructions , 1993, Workshop On Intentionality And Structure In Discource Relations.

[9]  Philip R. Cohen,et al.  Intentions in Communication , 1992, Language.

[10]  Pattie Maes,et al.  Designing autonomous agents: Theory and practice from biology to engineering and back , 1990, Robotics Auton. Syst..

[11]  W. Scott Neal Reilly,et al.  An Architecture for Action, Emotion, and Social Behavior , 1992, MAAMAW.

[12]  Norman I. Badler,et al.  Doing What You're Told: Following Task Instructions in Changing, but Hospitable Environments , 1992 .

[13]  Mark Steedman,et al.  Temporal Ontology and Temporal Reference , 1988, CL.

[14]  C. Sidner,et al.  Plans for Discourse , 1988 .

[15]  D. McDermott.  Transformational Planning of Reactive Behavior , 1992 .

[16]  Timothy W. Bickmore,et al.  A basic agent , 1990, Comput. Intell..

[17]  David J. Israel,et al.  Executions, motivations, and accomplishments , 1993 .

[18]  Barbara Di Eugenio,et al.  Understanding natural language instructions: a computational approach to purpose clauses , 1993 .

[19]  Cecile T. Balkanski,et al.  Actions, Beliefs and Intentions in Rationale Clauses and Means Clauses , 1992, AAAI.

[20]  Marcel Schoppers,et al.  Universal Plans for Reactive Robots in Unpredictable Environments , 1987, IJCAI.

[21]  Terry Winograd,et al.  Understanding natural language , 1974 .

[22]  Richard Alterman,et al.  Interaction, Comprehension, and Instruction Usage , 1991 .

[23]  Norman I. Badler,et al.  Simulating humans: computer graphics animation and control , 1993 .

[24]  Karen A. Schriver.  Teaching Writers to Anticipate Readers' Needs , 1992 .

[25]  Thomas Rist,et al.  The Design of Illustrated Documents as a Planning Task , 1993, AAAI Workshop on Intelligent Multimedia Interfaces.

[26]  Bonnie Webber,et al.  Plan recognition in understanding instructions , 1992 .

[27]  L. Suchman.  Plans and situated actions , 1987 .

[28]  David Chapman,et al.  What are plans for? , 1990, Robotics Auton. Syst..

[29]  Philip R. Cohen,et al.  Intentions in Communication , 1991, CL.

[30]  P. Dixon.  The processing of organizational and component step information in written directions , 1987 .

[31]  Breck Baldwin,et al.  Accommodating Context Change , 1992, ACL.

[32]  Barbara Di Eugenio,et al.  On the Interpretation of Natural Language Instructions , 1992, COLING.

[33]  Peter F. Patel-Schneider,et al.  Living wiht Classic: When and How to Use a KL-ONE-Like Language , 1991, Principles of Semantic Networks.

[34]  Christopher W. Geib,et al.  SodaJack: An Architecture For Agents That Search For And Manipulate Objects , 1994 .

[35]  P. Dixon.  The structure of mental plans for following directions. , 1987 .

[36]  Barbara Di Eugenio,et al.  Understanding Natural Language Instructions: The Case of Purpose Clauses , 1992, ACL.

[37]  Thomas Duffy,et al.  Plain language : principles and practice , 1991 .

[38]  Cecile Tiberghien Balkanski,et al.  Actions, beliefs and intentions in multi-action utterances , 1993 .

[39]  Anand S. Rao,et al.  An Abstract Architecture for Rational Agents , 1992, KR.

[40]  B. Webber,et al.  A Consequence of Incorporating Intentions in Means-end Planning* , 1993 .

[41]  Barbara Di Eugenio,et al.  Free Adjuncts in Natural Language Instructions , 1990, COLING.

[42]  Thomas W. Calvert,et al.  Composition of realistic animation sequences for multiple human figures , 1991 .

[43]  Earl David Sacerdoti,et al.  A Structure for Plans and Behavior , 1977 .

[44]  Steven K. Feiner,et al.  Automating the generation of coordinated multimedia explanations , 1991, Computer.

[45]  Nicholas Haddock,et al.  Computational models of incremental semantic interpretation , 1989 .

[46]  Norman I. Badler,et al.  Real-Time Control of a Virtual Human Using Minimal Sensors , 1993, Presence: Teleoperators & Virtual Environments.