Perceptive Actions in Tetris

First a word of justi cation. Motor actions which do not directly control sensor orientation are not usually deemed part of the perceptual process. When a child shakes, bites and throws an object, he or she is performing actions that can serve other functions; they need not be connected to perception. Hence they tend to be classi ed as shaking, biting or throwing: terms that are logically distinct from perception. But often actions are so well coordinated with the perceptual process that they interpenetrate perception, making it impossible to properly understand the perceptual system if they are ignored. J.J. Gibson was the rst psychologist to emphasize this mutuality [Gibson, 1950, Gibson, 1966]. His central concern was how the constant activity of an organism created a dynamic stream of information that was richer in action-relevant data than static images. By looking for regularities between actions and the rst or second derivatives of the changing optical input created by those actions, he noticed that it was possible to build action control systems that could regulate behaviour without consulting static images of the world. The classical example is the way measurement of looming can be harnessed to control the wing span of gannets, the avoidance of obstacles and so on, see also [Lee and Reddish, 1981]. Gibson made e orts to extend his analysis to more complex behaviour by introducing the concept of affordance. But the concept has proven too vague and ambiguous to deliver clear hypotheses for testing. The data which we have collected of 30 subjects playing Tetris provides a rigourous computational laboratory from which to study some of Gibson's ideas. It also allows us to study other ideas about action and perception unavailable to Gibson because of his aversion to computation and information processing. One of these is that certain forms of action can substitute for the kind of mental operations that occur in heuristic problem solving: particularly heuristically controlled generate and test methods. In what follow we shall give a brief introduction to some of our data and describe how these might make contact with the notions of a ordance and heuristic search.