A Structural Complexity Measure for Predicting Human Planning Performance

A Structural Complexity Measure for Predicting Human Planning Performance Marco Ragni, Felix Steffenhagen, Thomas Fangmeier {ragni, steffenhagen, fangmeier}@cognition.uni-freiburg.de Center for Cognitive Science, Friedrichstrase 50 University of Freiburg, Germany Abstract Humans have an impressive ability to solve even computation- ally complex problems. Limited cognitive processing capa- bilities, however, impede an exhaustive search of the problem space. Thus, planning problems of the same size may require a different cognitive effort. Formal complexity aspects are inher- ent to a problem and set computational limits that a solver must deal with. For a measure of cognitive complexity, operational aspects of human cognition must be taken into account. We present a structural complexity measure for predicting human planning performance. This measure is based on the number and connectedness of subgoals necessary to solve a problem. This measure is evaluated on the PSPACE-complete puzzle game Rush Hour and is able to capture empirically measured difficulty for this game. Keywords: Planning, Cognitive Complexity. Introduction Planned and rational behavior are daily aspect in everyday life. Planning can be defined as, the anticipation of action steps or “a procedure for achieving a particular goal or desired outcome” (Morris & Ward, 2005, p. 1). In computer science, one distinguishes between optimal and satisfiable planning. The goal of optimal planning is to find a shortest possible so- lutions for a problem, whereas the goal of satisfiable planning is to find a solution at all. In AI and Cognitive Science finding a solution is often represented as a search of the problem space (Russell & Norvig, 2003). The problem space is defined by the oper- ators and problem states. Due to limited cognitive process- ing resources, humans are not able to search the problem space exhaustively, i.e., they do not apply any operator on any state. Humans are, nonetheless, able to solve computa- tionally complex problems by chunking information, in order to reduce the problem representation, (Ellis & Siegler, 1994; Kotovsky, Hayes, & Simon, 1985) and by applying heuristic search strategies (Miller, Galanter, & Pribram, 1960). Planning problems have various characteristics. Problems can be non-transparent, have multiple goals, can be solv- able, well-defined, dynamic, or decomposable. Another im- portant issue is the domain of the problem. A first mea- sure of the difficulty of a planning problem is the minimum number of steps necessary to solve the problem. In AI the different degrees of difficulty for problems are mostly clas- sified according to the number of computing operations or the amount of memory required to solve a problem. For an overview of the complexity of planning tasks please refer to Helmert (2008). These measures are asymptotically with re- spect to worst case boundaries for increasing problem sizes (Papadimitriou, 1994). However, computational complexity measures do not integrate local problem structures. This is important for a more detailed measurement of problem diffi- culty, because problems with shorter solution length can be more difficult to solve for humans. If too many operations are necessary, most humans seem to become overstrained, i.e., they make significantly more er- rors, need more time, and even start to guess. The difficulty for humans in solving planning problems can differ with re- gards to solvability, optimality, and response times. This im- plies, that there must be further problem-inherent planning differences which influence the performance of humans. This aspect is important for explaining varying cognitive effort as it occurs in human problem solving. A cognitive complexity measure (a formal measure which is able to cap- ture the human planning complexity) must not only integrate formal aspects of complexity, but also particularities of the human reasoning and planning process, e.g., the abundant use of heuristics or preferred operations. We will define our cognitive complexity measure and eval- uate it on (spatial) permutation problems like Rush Hour 1 . This planning problem developed by Nob Yoshigahara is a game with a visual-spatial presentation which is well-defined, solvable, decomposable, not dynamic and has only one goal. Given these settings, the number of operations can be con- trolled systematically and measured precisely. These plan- ning problems have an initial state, an explicit goal state (e.g. where a certain relation must hold), and a number of un- derlying operations. Compared to Tower of London, Rush Hour has advantages, which guide the decision to use the lat- ter: the problem size is easier to adjust, it has two dimen- sional features, the number of interacting objects is higher, which increases the difficulties for human reasoning (branch- ing, counterintuitive moves), and Rush Hour is PSPACE- complete (Flake & Baum, 2002) and sufficiently complex for our purposes. It is also possible to generate highly chal- lenging problems. Thus, it is important to find parameters, which describe more precisely difficulties humans encounter in planning tasks as was possible in classical theoretical com- puter science In the following we first analyze (formal) requirements of a cognitive complexity measure to capture the average human planning process and introduce a first notion of a structural complexity measure. This is exemplified on the PSPACE- complete puzzle game Rush Hour. This structural complexity measure, although defined formally, is able to capture empir- ically measured difficulty for this game. Identified solution strategies and examples conclude the paper. 1 A complete description of RushHour can be found at http://www.thinkfun.com/instructions