The combination of vision and touch depends on spatial proximity.

The nervous system often combines visual and haptic information about object properties such that the combined estimate is more precise than with vision or haptics alone. We examined how the system determines when to combine the signals. Presumably, signals should not be combined when they come from different objects. The likelihood that signals come from different objects is highly correlated with the spatial separation between the signals, so we asked how the spatial separation between visual and haptic signals affects their combination. To do this, we first created conditions for each observer in which the effect of combination--the increase in discrimination precision with two modalities relative to performance with one modality--should be maximal. Then under these conditions, we presented visual and haptic stimuli separated by different spatial distances and compared human performance with predictions of a model that combined signals optimally. We found that discrimination precision was essentially optimal when the signals came from the same location, and that discrimination precision was poorer when the signals came from different locations. Thus, the mechanism of visual-haptic combination is specialized for signals that coincide in space.

[1]  M. Wertheimer Laws of organization in perceptual forms. , 1938 .

[2]  A. A. Mullin,et al.  Principles of neurodynamics , 1962 .

[3]  C. H. Coombs,et al.  Mathematical psychology : an elementary introduction , 1970 .

[4]  David G. Stork,et al.  Pattern Classification , 1973 .

[5]  Drew H. Abney,et al.  Journal of Experimental Psychology : Human Perception and Performance Influence of Musical Groove on Postural Sway , 2015 .

[6]  A. Treisman,et al.  Illusory conjunctions in the perception of objects , 1982, Cognitive Psychology.

[7]  K. A. Stevens Surface tilt (the direction of slant): A neglected psychophysical variable , 1983, Perception & psychophysics.

[8]  Jaap Van Brakel,et al.  Foundations of measurement , 1983 .

[9]  William Bialek,et al.  Statistics of Natural Images: Scaling in the Woods , 1993, NIPS.

[10]  A. John Mallinckrodt,et al.  Data Reduction and Error Analysis for the Physical Sciences , 1993 .

[11]  M. Landy,et al.  Measurement and modeling of depth cue combination: in defense of weak fusion , 1995, Vision Research.

[12]  A. Yuille,et al.  Bayesian decision theory and psychophysics , 1996 .

[13]  W. Prinzmetal,et al.  The phenomenology of attention, part 1: Color, location, orientation, and "clarity" , 1997 .

[14]  William Prinzmetal,et al.  The Phenomenology of Attention , 1997, Consciousness and Cognition.

[15]  M. Wertheimer,et al.  A source book of Gestalt psychology. , 1939 .

[16]  M. Kubovy,et al.  On the Lawfulness of Grouping by Proximity , 1998, Cognitive Psychology.

[17]  William Prinzmetal,et al.  Phenomenology of attention: I. Color, location, orientation, and spatial frequency. , 1998 .

[18]  C. von der Malsburg The what and why of binding: the modeler's perspective. , 1999, Neuron.

[19]  A. Roskies The Binding Problem , 1999, Neuron.

[20]  S. Shimojo,et al.  Illusions: What you see is what you hear , 2000, Nature.

[21]  Jonathan Barzilai,et al.  On the foundations of measurement , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[22]  W. Prinzmetal,et al.  The Phenomenology of Attention Part 1 : Color , Location , Orientation , and Spatial Frequency , 2001 .

[23]  Jeffrey S. Perry,et al.  Edge co-occurrence in natural images predicts contour grouping performance , 2001, Vision Research.

[24]  L M Ward,et al.  Multisensory integration and crossmodal attention effects in the human brain. , 2001, Science.

[25]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[26]  Jitendra Malik,et al.  A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[27]  M. Ernst,et al.  Humans integrate visual and haptic information in a statistically optimal fashion , 2002, Nature.

[28]  J. Elder,et al.  Ecological statistics of Gestalt laws for the perceptual organization of contours. , 2002, Journal of vision.

[29]  James M. Hillis,et al.  Combining Sensory Information: Mandatory Fusion Within, but Not Between, Senses , 2002, Science.

[30]  D. Wolpert,et al.  When Feeling Is More Important Than Seeing in Sensorimotor Adaptation , 2002, Current Biology.

[31]  S. Gepshtein,et al.  Viewing Geometry Determines How Vision and Haptics Combine in Size Perception , 2003, Current Biology.

[32]  M. Ernst,et al.  Feeling what you hear: auditory signals can modulate tactile tap perception , 2005, Experimental Brain Research.

[33]  D. Burr,et al.  The Ventriloquist Effect Results from Near-Optimal Bimodal Integration , 2004, Current Biology.

[34]  C. Spence,et al.  Exogenous spatial cuing studies of human crossmodal attention and multisensory integration , 2004 .

[35]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .