Interactive Grounding and Inference in Learning by Instruction

Learning by instruction is one of the most common forms of learning, and a number of research efforts have modeled the cognitive process of instruction following, with many successes. However, most computational models remain brittle with respect to the given instructions, and they lack the ability to adapt dynamically to variants of the instructions. This paper aims to illustrate modeling constructs designed to make instruction following more robust, including (1) more flexible grounding of language to execution, (2) processing of instructions that allows for inference of implicit instruction knowledge, and (3) dynamic, interactive clarification of instructions during both the learning and execution stages. Examples in the context of a paired-associates task and a visual-search task are discussed.

[1]  John E. Laird,et al.  Acquiring Grounded Representations of Words with Situated Interactive Instruction , 2012 .

[2]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[3]  Pat Langley,et al.  A Unified Cognitive Architecture for Physical Agents , 2006, AAAI.

[4]  John R. Anderson.  Interference: The relationship between response latency and response accuracy. , 1981 .

[5]  Niels Taatgen,et al.  The Multitasking Mind , 2010, Oxford series on cognitive models and architectures.

[6]  John E. Laird,et al.  The Soar Cognitive Architecture , 2012 .

[7]  Richard M. Young,et al.  The Role of Cognitive Architecture in Modeling the User: Soar's Learning Mechanism , 1997, Hum. Comput. Interact..

[8]  Nancy J. Cooke,et al.  Autonomous Intelligent Agents for Team Training , 2019, IEEE Intelligent Systems.

[9]  John E. Laird,et al.  Flexibly Instructable Agents , 1995, J. Artif. Intell. Res..

[10]  John E. Laird,et al.  Interactive Task Learning for Simple Games , 2014 .

[11]  John R. Anderson.  How Can the Human Mind Occur in the Physical Universe , 2007 .

[12]  Allen Newell,et al.  SOAR: An Architecture for General Intelligence , 1987, Artif. Intell..

[13]  John E. Laird,et al.  Grounding Language for Interactive Task Learning , 2017, RoboNLP@ACL.

[14]  Dario D. Salvucci.  Cognitive Code : An Embedded Approach to Cognitive Modeling , 2016 .

[15]  Allen Newell,et al.  Toward a Soar theory of taking instructions for immediate reasoning tasks , 1993 .

[16]  Dario D. Salvucci.  Integration and Reuse in Cognitive Skill Acquisition , 2013, Cogn. Sci..

[17]  Shaohua Yang,et al.  Language to Action: Towards Interactive Task Learning with Physical Agents , 2018, IJCAI.

[18]  Frank J. Lee,et al.  Production Compilation: A Simple Mechanism to Model Complex Skill Acquisition , 2003, Hum. Factors.

[19]  John R. Anderson,et al.  The acquisition of robust and flexible cognitive skills. , 2008, Journal of experimental psychology. General.

[20]  Niels Taatgen,et al.  Modeling Parallelization and Flexibility Improvements in Skill Acquisition: From Dual Tasks to Complex Dynamic Skills , 2005, Cogn. Sci..