A model of the combination of optic flow and extraretinal eye movement signals in primate extrastriate visual cortex: Neural model of self-motion from optic flow and extraretinal cues

The determination of the direction of heading from optic flow is a complicated task. To solve it the visual system complements the optic flow by non-visual information about the occurrence of eye movements. Psychophysical studies have shown that the need for this combination depends on the structure the visual scene. In a depth-rich visual environment motion parallax can be exploited to differentiate self-translation from eye rotation. In the absence of motion parallax, i.e. in the case of movement towards a frontoparallel plane, extraretinal signals are necessary for correct heading perception ([Warren and Hannon, 1990]). [Lappe and Rauschecker (1993b)] have proposed a model of visual heading detection that reproduces many of the psychophysical findings in the absence of extraretinal input and links them to properties of single neurons in the primate visual cortex. The present work proposes a neural network model that integrates extraretinal signals into this network. The model is compared with psychophysical and neurophysiological data from experiments in human and non-human primates. The combined visual/extraretinal model reproduces human behavior in the case of movement towards a frontoparallel plane. Single model neurons exhibit several similarities to neurons from the medial superior temporal (MST) area of the macaque monkey. Similar to MST cells ([Erickson and Thier, 1991]) they differentiate between self-induced visual motion that results from eye movements in a stationary environment, and real motion in the environment. The model predicts that this differentiation can also be achieved visually, i.e. without extraretinal input. Other simulations followed experiments by [Bradley et al. (1996)], in which flow fields were presented that simulated observer translation towards a frontoparallel plane plus an eye rotation. Similar to MST cells, model neurons shift their preference for the focus of expansion along the direction of the eye movement when extraretinal input is not available. They respond to the retinal location of the focus of expansion which is shifted by the eye movement. In the presence of extraretinal input the preference for the focus of expansion is largely invariant to eye movements and tied to the location of the focus of expansion with regard to the visual scene. The model proposes that extraretinal compensation for eye movements need not be perfect in single neurons to achieve accurate heading detection. It thereby shows that the incomplete compensation found in most MST neurons is sufficient to explain the psychophysical data.