Learning from Failures for Cognitive Flexibility

Learning from Failures for Cognitive Flexibility Dongkyu Choi (dongkyuc@uic.edu) Stellan Ohlsson (stellan@uic.edu) Department of Psychology University of Illinois at Chicago 1007 W Harrison Street (M/C 285), Chicago, IL 60607 USA Abstract existence of a very large number of potential discriminating features, leading to complex applicability conditions or large numbers of new rules or both; and the inability to identify potential discriminating features with a causal impact from those of accidental correlation. Cognitive flexibility is an important goal in the computational modeling of higher cognition. An agent operating in the world that changes over time should adapt to the changes and update its knowledge according to them. In this paper, we report our progress on implementing a constraint-based mechanism for learning from failures in a cognitive architecture, I CARUS . We review relevant features of the architecture, and describe the learning mechanism in detail. We also discuss the challenges encountered during the implementation and describe how we solved them. We then provide some experimental observations and conclude after a discussion on related and future work. Keywords: cognitive architecture, constraints, constraint violations, learning from failures, skill acquisition Introduction In computational models of higher cognition, it is impor- tant to simulate the broad human functionality that we call adaptability or flexibility. Cognitive flexibility is, of course, a multi-dimensional construct, but in this paper, we focus specifically on the ability of humans to act effectively when a familiar task environment is changing, thus rendering previ- ously learned skills ineffective or obsolete. Traditionally, researchers discussed two types of error cor- rection mechanisms for this problem. Weakening (Anderson, 1983, pp. 249–254) assumes that certain knowledge struc- tures like rules, skills, schemas, or chunks have strengths as- sociated with them, and it decreases the strength of the par- ticular structure that generates a negative outcome. However, actions themselves are not typically correct or incorrect, or appropriate or inappropriate. Instead, they are appropriate, correct or, useful in some situations but not in others. The goal of learning from failure is thus to distinguish between the class of situations in which a particular type of action will cause errors and the class of situations in which it does not. Weakening does not accomplish this, because lower strength makes an action less likely to be selected in all situations. Another mechanism proposed for error correction is dis- crimination (Langley, 1987). The key idea behind this con- tribution is to compare a situation with a positive outcome and another with a negative outcome to identify discriminat- ing features. If an action generates both positive and negative outcomes across multiple situations, the system identifies any features that were true in one situation but not in the other, and uses them to constrain the applicability of the action. But the computational discrimination mechanism also has several problems including: the lack of criterion for how many in- stances of either type are needed before a valid inference as to the discriminating features can be drawn; the possible In response, Ohlsson (1996) developed a constraint-based specialization mechanism for learning from negative out- comes. The production system implementation of the mech- anism overcomes most of the weaknesses of previous meth- ods. It assumes that the agent has access to some declara- tive knowledge in the form of constraints, which consist of an ordered pair with a relevance criterion and a satisfaction criterion. The system matches the relevance criteria of all constraints against the current state of the world on each cy- cle of its operation. For constraints with matching relevance conditions, the system also matches the satisfaction condi- tions. Satisfied constraints require no response, but violated constraints signal a failed expectation due to various reasons including a change in the world or erroneous knowledge. This constitutes a learning opportunity, and the system revises the current skill in such a way as to avoid violating the same constraint in the future. The computational problem involved here is to specify exactly how to revise the relevant skill when an error occurs, and the constraint-based specialization pro- vides a solution to this problem. Unlike weakening, the constraint-based approach identifies the specific class of situations in which an action is likely (or unlikely) to cause errors. It also differs from the discrim- ination method, and the mechanism does not carry out an uncertain, inductive inference. Instead, it computes a ratio- nally motivated revision to the current skill. However, these advantages were limited by a simplistic credit/blame attribu- tion algorithm and the lack of serious architectural supports like other learning mechanisms that can operate in parallel. In this paper, we adapt the constraint-based specialization mech- anism to a cognitive architecture, I CARUS , and address these limitations. The architecture features hierarchical knowledge structures, and it has a variety of well-developed capabilities including learning from positive outcomes (Langley & Choi, 2006). We first review the relevant features of the I CARUS architecture, and describe the constraint-based specialization mechanism in some detail. Then we identify the challenges we encountered during the implementation in I CARUS , with a particular attention to the credit assignment problem. Finally, we report some experimental observations with the system, and discuss related and future work.