The synthetic teammate project

The main objective of the Synthetic Teammate project is to develop language and task enabled synthetic agents capable of being integrated into team training simulations. To achieve this goal, the agents must be able to closely match human behavior. The initial application for the synthetic teammate research is creation of an agent able to perform the functions of a pilot for an Unmanned Aerial Vehicle (UAV) simulation as part of a three-person team. The agent, or synthetic teammate, is being developed in the ACT-R cognitive architecture. The major components include: language comprehension and generation, dialog management, agent-environment interaction, and situation assessment. Initial empirical results suggest that the agent-environment interaction is a good approximation to human behavior in the UAV environment, and we are planning further empirical tests of the synthetic teammate operating with human teammates. This paper covers the project’s modeling approach, challenges faced, progress made toward an integrated synthetic teammate, and lessons learned during development.

[1]  Stina Ericsson,et al.  Dynamic Optimisation of Information Enrichment in Dialogue , 2004 .

[2]  Jerry T. Ball,et al.  A Naturalistic, Functional Approach to Modeling Language Comprehension , 2008, AAAI Fall Symposium: Naturally-Inspired Artificial Intelligence.

[3]  Jerry Ball,et al.  Simplifying the Mapping from Referring Expression to Referent in a Conceptual Semantics of Reference , 2010 .

[4]  Nancy J. Cooke,et al.  Measuring team situation awareness in decentralized command and control environments , 2006, Ergonomics.

[5]  Vered Argaman,et al.  Against Repair-Based Reanalysis in Sentence Comprehension , 2003, Journal of psycholinguistic research.

[6]  Jerry T. Ball,et al.  A Cognitively Plausible Model of Language Comprehension , 2004 .

[7]  Jerry Ball,et al.  Toward a Large-Scale Model of Language Comprehension in ACT-R 6 , 2007 .

[8]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[9]  David E. Kieras,et al.  An Overview of the EPIC Architecture for Cognition and Performance With Application to Human-Computer Interaction , 1997, Hum. Comput. Interact..

[10]  James L. McClelland,et al.  A distributed, developmental model of word recognition and naming. , 1989, Psychological review.

[11]  Jerry T. Ball,et al.  A Bi-Polar Theory of Nominal and Clause Structure and Function , 2004 .

[12]  David D. Mcdonald.  A Rational Reconstruction of Genaro , 1999 .

[13]  David R. Traum,et al.  Negotiation over tasks in hybrid human-agent teams for simulation-based training , 2003, AAMAS '03.

[14]  Christian Lebiere,et al.  Between a rock and a hard place: cognitive science principles meet AI-hard problems : papers from the AAAI Spring Symposium , 2006 .

[15]  Nancy J. Cooke,et al.  Measuring Team Knowledge During Skill Acquisition of a Complex Task , 2001 .

[16]  A. Hollingworth,et al.  Thematic Roles Assigned along the Garden Path Linger , 2001, Cognitive Psychology.

[17]  Walter Kintsch,et al.  Toward a model of text comprehension and production. , 1978 .

[18]  David Klahr,et al.  Structure and process in alphabetic retrieval. , 1983 .

[19]  Alain Colmerauer,et al.  The birth of Prolog , 1996 .

[20]  Jerry T. Ball.  Can NLP Systems be a Cognitive Black Box? (Is Cognitive Science Relevant to AI Problems?) , 2006, AAAI Spring Symposium: Between a Rock and a Hard Place: Cognitive Science Principles Meet AI-Hard Problems.

[21]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: part 1.: an account of basic findings , 1988 .

[22]  Pat Langley,et al.  Ability, Breadth, and Parsimony in Computational Models of Higher-Order Cognition , 2008, Cogn. Sci..

[23]  Jerry R. Hobbs.  Ontological Promiscuity , 1985, ACL.

[24]  Milind Tambe,et al.  Intelligent Agents for Interactive Simulation Environments , 1995, AI Mag..

[25]  John R. Anderson,et al.  How Can the Human Mind Occur , 2007 .

[26]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[27]  Scott Douglass,et al.  Large Declarative Memories in ACT-R , 2009 .

[28]  G. Kempen,et al.  Syntactic structure assembly in human parsing: a computational model based on competitive inhibition and a lexicalist grammar , 2000, Cognition.

[29]  John R. Anderson,et al.  The role of examples and rules in the acquisition of a cognitive skill. , 1997, Journal of experimental psychology. Learning, memory, and cognition.

[30]  Frank E. Ritter,et al.  A User Modeling Design Tool Based on a Cognitive Architecture for Comparing Interfaces , 2002, CADUI.

[31]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[32]  C. Frith,et al.  Functional imaging of ‘theory of mind’ , 2003, Trends in Cognitive Sciences.

[33]  Andreas Abecker,et al.  Between a Rock and a Hard Place : Cognitive Science Principles Meet AI Hard Problems , 2006 .

[34]  Marilyn A. Walker,et al.  Generation and evaluation of user tailored responses in multimodal dialogue , 2004 .

[35]  Randolph M. Jones,et al.  Automated Intelligent Pilots for Combat Flight Simulation , 1998, AI Mag..

[36]  P. Boersma,et al.  Empirical Tests of the Gradual Learning Algorithm , 2001, Linguistic Inquiry.

[37]  Jerry R. Hobbs.  DISCOURSE AND INFERENCE , 2003 .

[38]  Sebastian Varges.  Overgeneration and Ranking for Spoken Dialogue Systems , 2006, INLG.

[39]  David E. Kieras,et al.  Towards a Practical GOMS Model Methodology for User Interface Design , 1988 .

[40]  Stuart M. Rodgers,et al.  Integrating ACT-R and Cyc in a large-scale model of language comprehension for use in intelligent agents , 2004 .

[41]  N. Pearlmutter,et al.  Constraints on sentence comprehension , 1998, Trends in Cognitive Sciences.

[42]  Mark Steedman,et al.  Interaction with context during human sentence processing , 1988, Cognition.

[43]  Julie C. Sedivy,et al.  Subject Terms: Linguistics Language Eyes & eyesight Cognition & reasoning , 1995 .

[44]  Uwe Reyle,et al.  From discourse to logic , 1993 .

[45]  Wayne Zachary,et al.  Using a Community of Intelligent Synthetic Entities to Support Operational Team Training , 2001 .

[46]  John R. Anderson.  How Can the Human Mind Occur in the Physical Universe , 2007 .

[47]  Jerry T. Ball,et al.  Construction Driven Language Processing , 2005 .

[48]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[49]  P. Bello,et al.  Some Empirical Results Concerning Deontic Reasoning: Models, Schema, or Both? , 2005 .

[50]  Jerry Thomas Ball,et al.  PM, propositional model, a computational psycholinguistic model of language comprehension based on a relational analysis of written English , 1992 .

[51]  David E. Kieras,et al.  Automating interface evaluation , 1994, CHI '94.

[52]  Michael D. Byrne,et al.  ACT-R/PM and menu selection: applying a cognitive architecture to HCI , 2001, Int. J. Hum. Comput. Stud..

[53]  Michael D. Byrne,et al.  Using Computational Cognitive Modeling to Diagnose Possible Sources of Aviation Error , 2005 .

[54]  David R. Traum,et al.  Discourse Obligations in Dialogue Processing , 1994, ACL.

[55]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[56]  Uwe Reyle,et al.  From Discourse to Logic - Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory , 1993, Studies in linguistics and philosophy.

[57]  R. Langacker.  Foundations of cognitive grammar , 1983 .

[58]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[59]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[60]  Nancy J. Cooke,et al.  Team cognition in experienced command-and-control teams. , 2007, Journal of experimental psychology. Applied.

[61]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[62]  John R. Anderson,et al.  Simulating adaptive communication , 2000 .

[63]  P. Smolensky,et al.  Optimality Theory: Constraint Interaction in Generative Grammar , 2004 .

[64]  Walter Kintsch,et al.  Comprehension: A Paradigm for Cognition , 1998 .