Neural mechanisms of three-dimensional vision

We can see things in three dimensions because the visual system re-constructs the three-dimensional (3D) configurations of objects from their two-dimensional (2D) images projected onto the retinas. The purpose of this paper is to give an overview of the psychological background and recent physiological findings concerning three-dimensional vision. Psychophysical and computational studies have suggested that in the visual system the 3D surface orientation is first estimated independently from individual depth cues--such as binocular disparity, as well as various monocular cues including texture gradients--and then the information from these different depth cues is integrated to construct a generalized representation of the 3D surface geometry. Neurons involved in low-level disparity processing, or the detection of local absolute disparity, were found mainly in the occipital cortex, whereas neurons involved in high-level disparity processing, or the reconstruction of 3D surface orientation through the computation of disparity gradients, were found mainly in the parietal area caudal intraparietal sulcus (CIP). Neurons sensitive to texture gradients, which is one of the major monocular cues, were also found in CIP. The majority of these neurons were sensitive to disparity gradients as well, suggesting their involvement in the computation of 3D surface orientation. In CIP, neurons sensitive to multiple depth cues were widely distributed together with those sensitive to a specific depth cue, suggesting CIP's involvement in the integration of depth information from different sources. In addition, human and monkey imaging studies have indicated convergence of multiple depth cues in CIP. These neurophysiological findings suggest that CIP plays a critical role in 3D vision by constructing a generalized representation of the 3D surface geometry of objects.