Taking the brain serious: introduction to the special issue on integration in and across perception and action

Our world consists of object and events. As an almost defining criterion of how an object is constituted, the features of an object are all attached to each other: the redness of the cherry is exactly where I find its roundness and the softness of its skin, and all these features are there at the same point in time. Things get a bit more complicated if we consider larger objects, like a sailing ship—where the whiteness may be restricted to the sail, and temporally extended events, where features may gradually change over time. But the basic organizing structure is always the same, namely, things that belong together share spatiotemporal coordinates. This observation may seem trivial, but it is in fact striking when one considers that our brain has a very different organizational structure. One important difference is that the brain codes the different features of an object or event in different modules: color is coded in color maps, aspects of shapes in shape maps, motion in motion maps, pitch in pitch maps, and so on (e.g., DeYoe & Van Essen, 1988; Schreiner, 1995). Some of these maps are spatially organized, and thus retain at least this aspect of how our world is organized, but the sheer multiplicity of representational maps demonstrates that properties that are perceived together in the world are not represented together in the brain. A second important difference concerns the temporal aspect of perception. The computation and transmission of different features within the same modality and in different modalities varies widely, so that the time at which a representational code is activated is an unreliable estimate of when the represented event occurred in the world (Zeki, 1993). The implication is that things that occur at the same point in time are not reliably represented by temporal aspects of their representations. Our brain is thus facing the problem that the spatiotemporal characteristics of objects and events in our world are distorted in the process of coding them, so that representations of things cannot be mere copies of their attributes in the world. But how else does our brain keep things together? This question has become known as the binding problem, which was introduced to cognitive psychology by Alan Allport (e.g., Allport, Tipper, & Chmiel, 1985) and Anne Treisman (e.g., Treisman & Gelade, 1980). As these authors pointed out, distributed representations such as those apparently preferred by the human brain need some mechanism that cross-links or binds the representational codes that refer to the same object or event. Assume, for instance, you are facing a green square and a red circle. Processing them will activate the codes , , , and in your perceptual system, so that some mechanism must sort those codes in such a way that will be related to but not . The binding problem has been addressed in numerous ways and we will not be able to discuss them in detail (cf., Treisman, 1996). Yet it is interesting to see that the most accepted solutions to the binding problem suggest that the brain in some sense tries to imitate or reconstruct the spatiotemporal organization of the world. Spatial organization is reintroduced by restricting a hypothetical attentional spotlight to a single object and/or location in space, so that only those feature codes that belong to this object and/or the location it occupies get (strongly) activated (e.g., van der Heijden, 1992; Treisman & Gelade, 1980). Temporal organization is reintroduced by creating temporal integration windows (implied but commonly not discussed by space-related models), during which the available information about the attended object, or at the attended location, is sampled. The length of the assumed integration window B. Hommel (&) Department of Psychology, Cognitive Psychology Unit, Leiden University, P.O. Box 9555, 2300 Leiden, The Netherlands E-mail: hommel@fsw.leidenuniv.nl