Multisensory numerosity judgments for visual and tactile stimuli

To date, numerosity judgments have been studied only under conditions of unimodal stimulus presentation. It is therefore unclear whether the same limitations on correctly reporting the number of unimodal visual or tactile stimuli presented in a display might be expected under conditions in which participants have to count stimuli presented simultaneously in two or more different sensory modalities. In Experiment 1, we investigated numerosity judgments using both unimodal and bimodal displays consisting of one to six vibrotactile stimuli (presented over the body surface) and one to six visual stimuli (seen on the body via mirror reflection). Participants had to count the number of stimuli regardless of their modality of presentation. Bimodal numerosity judgments were significantly less accurate than predicted on the basis of an independent modality-specific resources account, thus showing that numerosity judgments might rely on a unitary amodal system instead. The results of a second experiment demonstrated that divided attention costs could not account for the poor performance in the bimodal conditions of Experiment 1. We discuss these results in relation to current theories of cross-modal integration and to the cognitive resources and/or common higher order spatial representations possibly accessed by both visual and tactile stimuli.

[1]  J. V. Erp,et al.  Vibrotactile in-vehicle navigation system , 2004 .

[2]  Hong Z. Tan,et al.  The Body Surface as a Communication System: The State of the Art after 50 Years , 2007, PRESENCE: Teleoperators and Virtual Environments.

[3]  C T WHITE,et al.  Temporal numerosity: IV. A comparison of the major senses. , 1959, Journal of Experimental Psychology.

[4]  Y. Rossetti,et al.  Ameliorating neglect with prism adaptation: visuo-manual and visuo-verbal measures , 2002, Neuropsychologia.

[5]  Michael J. Wright,et al.  Limitations for Change Detection in Multiple Gabor Targets , 2000 .

[6]  E C Lechelt,et al.  Pulse Number Discrimination in Tactile Spatio-Temporal Patterns , 1974, Perceptual and motor skills.

[7]  C. Spence,et al.  Modality-specific auditory and visual temporal processing deficits , 2002, The Quarterly journal of experimental psychology. A, Human experimental psychology.

[8]  John Duncan,et al.  Within-modality and cross-modality attentional blinks in a simple discrimination task , 2006, Perception & psychophysics.

[9]  J. Armington,et al.  Temporal numerosity. II. Evidence for central factors influencing perceived number. , 1953, Journal of experimental psychology.

[10]  Giuseppe di Pellegrino,et al.  Neuropsychological Evidence of an Integrated Visuotactile Representation of Peripersonal Space in Humans , 1998, Journal of Cognitive Neuroscience.

[11]  Jon Driver,et al.  Attentional competition between modalities: extinction between touch and vision after right hemisphere damage , 1997, Neuropsychologia.

[12]  E. M. Jensen,et al.  The subitizing and counting of visually presented fields of dots. , 1950 .

[13]  P Viviani,et al.  Choice reaction times for temporal numerosity. , 1979, Journal of experimental psychology. Human perception and performance.

[14]  C. Spence,et al.  Seeing Your Own Touched Hands in a Mirror Modulates Cross-Modal Interactions , 2002, Psychological science.

[15]  J R Hennessy Cutaneous Sensitivity Communications1 , 1966, Human factors.

[16]  F. Gregory Ashby,et al.  Subitizing: Magical numbers or mere superstition? , 1992, Psychological research.

[17]  Daniel Gopher,et al.  On the Economy of the Human Processing System: A Model of Multiple Capacity. , 1977 .

[18]  H. Ono,et al.  Depth perception as a function of motion parallax and absolute-distance information. , 1986, Journal of experimental psychology. Human perception and performance.

[19]  T. Hirahara,et al.  One, two, many—Judging the number of concurrent talkers. , 1996 .

[20]  J. G. Hollands,et al.  Engineering Psychology and Human Performance , 1984 .

[21]  S Lakatos,et al.  Time—distance relations in shifting attention between locations on one’s body , 1997, Perception & psychophysics.

[22]  B. B. Morgan,et al.  Masking of Cutaneous Sensations in Multiple Stimulus Presentations , 1965, Perceptual and motor skills.

[23]  Z. Pylyshyn,et al.  What enumeration studies can show us about spatial attention: evidence for limited capacity preattentive processing. , 1993, Journal of experimental psychology. Human perception and performance.

[24]  E. Miller,et al.  Analog Numerical Representations in Rhesus Monkeys: Evidence for Parallel Processing , 2004, Journal of Cognitive Neuroscience.

[25]  F. A. Geldard,et al.  MULTIPLE CUTANEOUS STIMULATION: THE DISCRIMINATION OF VIBRATORY PATTERNS. , 1965, The Journal of the Acoustical Society of America.

[26]  P. Haggard,et al.  Facilitated Processing of Visual Stimuli Associated with the Body , 2004, Perception.

[27]  Perceived number and evoked cortical potentials. , 1968 .

[28]  G Vallar,et al.  Anatomical correlates of visual and tactile extinction in humans: a clinical CT scan study. , 1994, Journal of neurology, neurosurgery, and psychiatry.

[29]  C. Spence,et al.  Attention and the crossmodal construction of space , 1998, Trends in Cognitive Sciences.

[30]  C. Spence,et al.  Exogenous spatial cuing studies of human crossmodal attention and multisensory integration , 2004 .

[31]  Brian Butterworth,et al.  Are Subitizing and Counting Implemented as Separate or Functionally Overlapping Processes? , 2002, NeuroImage.

[32]  G. Rizzolatti,et al.  Afferent properties of periarcuate neurons in macaque monkeys. I. Somatosensory responses , 1981, Behavioural Brain Research.

[33]  Robert Edward Taubman,et al.  Studies in Judged Number: II. The Judgment of Visual Number , 1950 .

[34]  G. Mandler,et al.  Subitizing: an analysis of its component processes. , 1982, Journal of experimental psychology. General.

[35]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[36]  C. Spence,et al.  Visual capture of apparent limb position influences tactile temporal order judgments , 2005, Neuroscience Letters.

[37]  F. Colavita Human sensory dominance , 1974 .

[38]  Garner Wr The accuracy of counting repeated short tones. , 1951 .

[39]  Hong Z. Tan,et al.  Tactile change detection , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[40]  Ryan M. Traylor,et al.  A Haptic Back Display for Attentional and Directional Cueing , 2003 .

[41]  Kazuo Hiraki,et al.  Auditory-visual intermodal matching of small numerosities in 6-month-old infants. , 2005, Developmental science.

[42]  Hendrik A. H. C. van Veen,et al.  Tactile Information Presentation in the Cockpit , 2000, Haptic Human-Computer Interaction.

[43]  E. Lechelt Spatial numerosity discrimination as contingent upon sensory and extrinsic factors , 1971 .

[44]  E. Spelke,et al.  Numerical abstraction by human infants , 1990, Cognition.

[45]  Edwin A. Fleishman,et al.  Psychology and human performance. , 1959 .

[46]  C T WHITE,et al.  Temporal numerosity. III. Auditory perception of number. , 1954, Journal of experimental psychology.

[47]  F. Campbell,et al.  The Dependence of the Visual Numerosity Limit on Orientation, Colour, and Grouping in the Stimulus , 1976, Perception.

[48]  W R GARNER The accuracy of counting repeated short tones. , 1951, Journal of experimental psychology.

[49]  E. Lechelt Temporal numerosity discrimination: intermodal comparisons revisited. , 1975, British journal of psychology.

[50]  R Rothkegel,et al.  Subitizing and its subprocesses , 2000, Psychological research.

[51]  C. Spence,et al.  Numerosity Judgments for Tactile Stimuli Distributed over the Body Surface , 2006, Perception.

[52]  Z. Pylyshyn,et al.  Why are small and large numbers enumerated differently? A limited-capacity preattentive stage in vision. , 1994, Psychological review.

[53]  Jon Driver,et al.  Crossmodal Spatial Attention: Evidence from Human Performance , 2004 .

[54]  Georg v. Békésy,et al.  Neural Funneling along the Skin and between the Inner and Outer Hair Cells of the Cochlea , 1959 .

[55]  Jon Driver,et al.  Multiple resources and multimodal interface design , 1999 .

[56]  Shin'ya Nishida,et al.  Auditory localization aftereffects. , 1996 .

[57]  Z W Pylyshyn,et al.  Tracking multiple independent targets: evidence for a parallel tracking mechanism. , 1988, Spatial vision.

[58]  C. Spence,et al.  Spatial constraints on visual-tactile cross-modal distractor congruency effects , 2004, Cognitive, affective & behavioral neuroscience.

[59]  J W Hill,et al.  Processing of tactual and visual point stimuli sequentially presented at high rates. , 1971, Journal of experimental psychology.

[60]  J. Fodor The Modularity of mind. An essay on faculty psychology , 1986 .

[61]  Hong Z. Tan,et al.  When visual transients impair tactile change detection: A novel case of crossmodal change blindness? , 2006, Neuroscience Letters.

[62]  E. L. Kaufman,et al.  The discrimination of visual number. , 1949, The American journal of psychology.

[63]  D J Newman,et al.  A tactile display for international space station (ISS) extravehicular activity (EVA). , 2000, Aviation, space, and environmental medicine.

[64]  Gavriel Salvendy,et al.  Handbook of human factors. , 1987 .

[65]  Felice L. Bedford,et al.  Analysis of a constraint on perception, cognition, and development: one object, one place, one time. , 2004, Journal of experimental psychology. Human perception and performance.

[66]  Hong Z. Tan,et al.  Using spatial vibrotactile cues to direct visual attention in driving scenes , 2005 .

[67]  N. Cowan The magical number 4 in short-term memory: A reconsideration of mental storage capacity , 2001, Behavioral and Brain Sciences.

[68]  Christopher D. Wickens,et al.  The Structure of Attentional Resources , 1980 .

[69]  Maryanne Martin,et al.  Attention to words in different modalities: four-channel presentation with physical and semantic selection. , 1980, Acta psychologica.

[70]  Elisabetta Làdavas,et al.  Seeing where your hands are , 1997, Nature.

[71]  E. Macaluso,et al.  Spatial attention and crossmodal interactions between vision and touch , 2001, Neuropsychologia.

[72]  C T White,et al.  Temporal numerosity and the psychological unit of duration. , 1963, Psychological monographs.

[73]  R. Rafal,et al.  A systematic study of visual extinction. Between- and within-field deficits of attention in hemispatial neglect. , 2000, Brain : a journal of neurology.

[74]  Robert Edward Taubman,et al.  Studies in Judged Number: I. The Judgment of Auditory Number , 1950 .

[75]  J. Cantlon,et al.  Shared System for Ordering Small and Large Numbers in Monkeys and Humans , 2006, Psychological science.

[76]  Andreas Nieder,et al.  The Number Domain— Can We Count on Parietal Cortex? , 2004, Neuron.

[77]  Joos Vos,et al.  Effect on numerosity judgment of grouping of tones by auditory channels , 1979, Perception & psychophysics.

[78]  Tony J. Simon,et al.  Computational evidence for the subitizing phenomenon as an emergent property of the human cognitive architecture , 2000, Cogn. Sci..

[79]  C. Spence,et al.  Multisensory prior entry. , 2001, Journal of experimental psychology. General.

[80]  Elizabeth M Brannon,et al.  The multisensory representation of number in infancy. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[81]  C. Spence,et al.  Visual dominance and attention: The Colavita effect revisited , 2007, Perception & psychophysics.

[82]  P. Bertelson,et al.  The psychology of multimodal perception , 2004 .

[83]  F. Campbell,et al.  The Magic Number 4 ± 0: A New Look at Visual Numerosity Judgements , 1976, Perception.

[84]  C. Packer,et al.  Roaring and numerical assessment in contests between groups of female lions, Panthera leo , 1994, Animal Behaviour.

[85]  C. Spence,et al.  The cost of expecting events in the wrong sensory modality , 2001, Perception & psychophysics.

[86]  C. Spence,et al.  The failure to detect tactile change: A tactile analogue of visual change blindness , 2006, Psychonomic bulletin & review.

[87]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .

[88]  C T WHITE,et al.  Temporal numerosity: I. Perceived number as a function of flash number and rate. , 1952, Journal of experimental psychology.

[89]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[90]  Christopher D. Wickens,et al.  Multiple resources and performance prediction , 2002 .

[91]  Ronald A. Rensink Change detection. , 2002, Annual review of psychology.

[92]  T. B. Posey,et al.  Numerosity Discrimination of Tactile Stimuli , 1976, Perceptual and motor skills.

[93]  Michael S Vitevitch,et al.  Change deafness: the inability to detect changes between two voices. , 2003, Journal of experimental psychology. Human perception and performance.

[94]  C. Spence,et al.  Attentional limitations in processing sequentially presented vibrotactile targets , 2002, Perception & psychophysics.

[95]  W. STANLEY JEVONS,et al.  The Power of Numerical Discrimination , 1871, Nature.

[96]  S. Dehaene,et al.  Interactions between number and space in parietal cortex , 2005, Nature Reviews Neuroscience.

[97]  Anders M. Dale,et al.  Optical Imaging of a Tactile Illusion in Area 3b of the Primary Somatosensory Cortex , 2003 .

[98]  E. Macaluso,et al.  Functional imaging of crossmodal spatial representations and crossmodal spatial attention , 2004 .

[99]  C D Wickens,et al.  Codes and Modalities in Multiple Resources: A Success and a Qualification , 1988, Human factors.

[100]  A. Kleinschmidt,et al.  A Supramodal Number Representation in Human Intraparietal Cortex , 2003, Neuron.

[101]  Stanislas Dehaene,et al.  Development of Elementary Numerical Abilities: A Neuronal Model , 1993, Journal of Cognitive Neuroscience.

[102]  S L Wolf,et al.  Doing It with Mirrors: A Case Study of a Novel Approach to Neurorehabilitation , 2000, Neurorehabilitation and neural repair.

[103]  G. A. Miller THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[104]  Andrew Simpson,et al.  Subitizing in Tactile Perception , 2006, Psychological science.

[105]  Donald L. Fisher,et al.  Central capacity limits in consistent mapping, visual search tasks: Four channels or more? , 1984, Cognitive Psychology.

[106]  J. D. Balakrishnan,et al.  Is subitizing a unique numerical ability? , 1991, Perception & psychophysics.

[107]  J.B.F. van Erp,et al.  A multi-purpose tactile vest for astronauts in the international space station , 2003 .

[108]  E. P. Gardner,et al.  Sensory funneling. II. Cortical neuronal representation of patterned cutaneous stimuli. , 1972, Journal of neurophysiology.

[109]  Andreas Nieder,et al.  A parieto-frontal network for visual numerical information in the monkey. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[110]  Influence of an Irrelevant Stimulus Attribute on Numerosity Judgments , 1965, Perceptual and motor skills.

[111]  C. Koch,et al.  Competition and selection during visual processing of natural scenes and objects. , 2003, Journal of vision.

[112]  Elizabeth M. Brannon,et al.  Monkeys Match the Number of Voices They Hear to the Number of Faces They See , 2005, Current Biology.