An Asic-Chip for Stereoscopic Depth Analysis in Video-Real-Time Based on Visual Cortical Cell Behavior

In a stereoscopic system both eyes or cameras have a slightly different view. As a consequence small variations between the projected images exist ("disparities") which are spatially evaluated in order to retrieve depth information. We will show that two related algorithmic versions can be designed which recover disparity. Both approaches are based on the comparison of filter outputs from filtering the left and the right image. The difference of the phase components between left and right filter responses encodes the disparity. One approach uses regular Gabor filters and computes the spatial phase differences in a conventional way as described already in 1988 by Sanger. Novel to this approach, however, is that we formulate it in a way which is fully compatible with neural operations in the visual cortex. The second approach uses the apparently paradoxical similarity between the analysis of visual disparities and the determination of the azimuth of a sound source. Animals determine the direction of the sound from the temporal delay between the left and right ear signals. Similarly, in our second approach we transpose the spatially defined problem of disparity analysis into the temporal domain and utilize two resonators implemented in the form of causal (electronic) filters to determine the disparity as local temporal phase differences between the left and right filter responses. This approach permits video real-time analysis of stereo image sequences (see movies at http://www.neurop.ruhr-uni-bochum.de/Real- Time-Stereo) and a FPGA-based PC-board has been developed which performs stereo-analysis at full PAL resolution in video real-time. An ASIC chip will be available in March 2000.

[1]  J. Daugman.  Two-dimensional spectral analysis of cortical receptive field profiles , 1980, Vision Research.

[2]  D. Pollen,et al.  Phase relationships between adjacent simple cells in the visual cortex. , 1981, Science.

[3]  D. Pollen,et al.  Spatial computation performed by simple and complex cells in the visual cortex of the cat , 1982, Vision Research.

[4]  E H Adelson,et al.  Spatiotemporal energy models for the perception of motion. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[5]  M. Konishi,et al.  Neural map of interaural phase difference in the owl's brainstem. , 1986, Proceedings of the National Academy of Sciences of the United States of America.

[6]  J. P. Jones,et al.  The two-dimensional spatial structure of simple receptive fields in cat striate cortex. , 1987, Journal of neurophysiology.

[7]  D. Ferster.  Spatially opponent excitation and inhibition in simple cells of the cat visual cortex , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  I. Ohzawa,et al.  Stereoscopic depth discrimination in the visual cortex: neurons ideally suited as disparity detectors. , 1990, Science.

[9]  David J. Fleet,et al.  Phase-based disparity measurement , 1991, CVGIP Image Underst..

[10]  Gregory C. DeAngelis,et al.  Depth is encoded in the visual cortex by a specialized receptive field structure , 1991, Nature.

[11]  D. Pollen,et al.  Interneuronal interaction between members of quadrature phase and anti-phase pairs in the cat's visual cortex , 1992, Vision Research.

[12]  Hermann Wagner,et al.  Disparity-sensitive cells in the owl have a characteristic disparity , 1993, Nature.

[13]  David J. Fleet,et al.  Stability of Phase Information , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  David J. Fleet,et al.  Neural encoding of binocular disparity: Energy models, position shifts and phase shifts , 1996, Vision Research.

[15]  Ning Qian,et al.  Physiological computation of binocular disparity , 1997, Vision Research.

[16]  I. Ohzawa,et al.  Encoding of binocular disparity by complex cells in the cat's visual cortex. , 1996, Journal of neurophysiology.

[17]  Florentin Wörgötter,et al.  Machine Vision and Applications Manuscript Nr. Performance of Phase-based Algorithms for Disparity Estimation , 2022 .

[18]  F. Wörgötter,et al.  A possible basic cortical microcircuit called ’’cascaded inhibition’’Results from cortical network models and recording experiments from striate simple cells , 1998, Experimental Brain Research.

[19]  Florentin Wörgötter,et al.  How to “hear” visual disparities: real-time stereoscopic spatial depth analysis using temporal resonance , 1998, Biological Cybernetics.