The Merging of the Senses

The Merging of the Senses is primarily an overview of research on the integration of sensory inputs in the deep layers of the superior colliculus of the brain, and the consequences of this integration for orienting responses in animals. It is an excellent overview, lucid, cogent, and eminently readable. The first two chapters attempt to place the reported research in a broader context. Chapter 1 addresses some intriguing phenomena related to sensory interactions in humans; these include the “ventriloquism effect” (an instance of the more general phenomenon of “visual capture’’), synesthesia (experiencing input from one modality as input from another), and cross-modal matching. However, these topics are only considered briefly. Chapter 2 reviews some instances of sensory integration at various levels of the phylogenetic scale, ranging from protozoan to vertebrate. Again the coverage of each topic is brief, though interesting. It attests to the clarity of the presentation that the present reviewer-a perceptual and cognitive psychologist-found himself reading with genuine interest about the cell membrane of the paramecium. With Chapter 3, the book begins to deal in earnest with its central theme. Chapters 3 to 5 are concerned with the functional anatomy of the superior colliculus. Efferent and afferent pathways are reviewed. Inputs from the visual, auditory, somatosensory, and nociceptive (pain) sensory systems are documented. Motor maps and gaze fields are discussed. Interactions between the colliculus and cortex are explained. These chapters will provide a reader unfamiliar with the superior colliculus with a first class introduction to this structure, and can serve as a valuable review for readers previously acquainted with this material. Chapters 6-10 describe how the sensory modalities interact in governing the responses of collicular sensory and premotor neurons. Many of the investigations reported were carried out by the authors of the book and their colleagues, and involve single cell recording in the superior colliculus of the cat. However, experiments in reptiles, birds, and primates attest to the generality of the conclusions drawn. The research described demonstrates the presence in the colliculus of visual (retinotopic), auditory, and somatosensory maps in spatial register. These maps are largely formed by multimodal neurons (e.g., neurons with combined visual-auditorysomatosensory receptive fields). Sensory inputs to these multimodal neurons combine in a multiplicative fashion: visual and auditory stimuli in spatial register can produce a neural response far larger than the sum of a neuron’s responses to the stimuli presented singly. On the other hand, when stimuli are presented out of spatial register, the result can be inhibition-the neuron’s response is smaller than its response to the stimuli presented singly. Chapter 11 describes experiments carried out by the authors to demonstrate the behavioral consequences of multisensory integration. Cats are initially trained to approach a visual or auditory cue. When the intensity of each of these stimuli is too low to elicit a behavioral response, robust approach responses can be obtained when they are presented together in spatial register. The authors argue that the observed response enhancement is well beyond that predicted by statistical methods. In contrast, when auditory and visual stimuli are presented out of spatial register, a visual cue that normally evokes a response often fails to do so, despite the fact that auditory stimulus evokes no response when presented alone. While an evaluation of these data in terms of the performance gains predicted from probability summation would be desirable, the effects reported are robust and the fit between the neurophysiological and behavioral data seems compelling. In general, the behavioral studies are extremely effective in demonstrating the probable import of the single-cell findings. Chapter 12 deals with the problem of how the various sensory maps in the colliculus are brought into and kept in alignment with each other. Developmentally, alignment appears to depend on experience with multimodal stimulation. It is less clear how the various maps maintain alignment despite eye movements, and solutions may be species specific. In the cat, behavioral mechanisms seem to be present that are designed to keep the eyes from deviating for more than short periods from straight ahead with respect to the head. In monkeys, there is evidence that the collicular auditory sensory map actually shifts in response to eye movements. In the final chapter, the authors once again move beyond the superior colliculus. Preliminary results in two multisensory cortical area of the cat brain, the anterior ectosylvian sulcus (AES) and lateral suprasylvian sulcus