A computational theory of visual surface interpolation.

Computational theories of structure-from-motion and stereo vision only specify the computation of three-dimensional surface information at special points in the image. Yet the visual perception is clearly of complete surfaces. To account for this a computational theory of the interpolation of surfaces from visual information is presented. The problem is constrained by the fact that the surface must agree with the information from stereo or motion correspondence, and not vary radically between these points. Using the image irradiance equation, an explicit form of this surface consistency constraint can be derived. To determine which of two possible surfaces is more consistent with the surface consistency constraint, one must be able to compare the two surfaces. To do this, a functional from the space of possible functions to the real numbers is required. In this way, the surface most consistent with the visual information will be that which minimizes the functional. To ensure that the functional has a unique minimal surface, conditions on the form of the functional are derived. In particular, if the functional is a complete semi-norm that satisfies the parallelogram law, or the space of functions is a semi-Hilbert space and the functional is a semi-inner product, then there is a unique (to within possibly an element of the null space of the functional) surface that is most consistent with the visual information. It can be shown, based on the above conditions plus a condition of rotational symmetry, that there is a vector space of possible functionals that measure surface consistency, this vector space being spanned by the functional of quadratic variation and the functional of square Laplacian. Arguments based on the null spaces of the respective functionals are used to justify the choice of the quadratic variation as the optimal functional. Possible refinements to the theory, concerning the role of discontinuities in depth and the effects of applying the interpolation process to scenes containing more than one object, are discussed.

[1]  C. Wheatstone.  XVIII. Contributions to the physiology of vision. —Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision , 1962, Philosophical Transactions of the Royal Society of London.

[2]  H J Howard,et al.  A Test for the Judgment of Distance. , 1919, Transactions of the American Ophthalmological Society.

[3]  W. Miles.  Movement Interpretations of the Silhouette of a Revolving Fan , 1931 .

[4]  L. S. Woodburne.  The effect of a constant visual angle upon the binocular discrimination of depth differences , 1934 .

[5]  R. Courant,et al.  Methods of Mathematical Physics , 1962 .

[6]  R. N. Berry.  Quantitative relations among vernier, real depth, and stereoscopic depth acuities. , 1948, Journal of experimental psychology.

[7]  B. O'Brien,et al.  Vision and resolution in the central retina. , 1951, Journal of the Optical Society of America.

[8]  H. Wallach,et al.  The kinetic depth effect. , 1953, Journal of experimental psychology.

[9]  B. Julesz.  Binocular depth perception of computer-generated patterns , 1960 .

[10]  G. Johansson.  PERCEPTION OF MOTION AND CHANGING FORM: A study of visual perception from continuous transformations of a solid angle of light at the eye , 1964 .

[11]  Berthold K. P. Horn.  SHAPE FROM SHADING: A METHOD FOR OBTAINING THE SHAPE OF A SMOOTH OPAQUE OBJECT FROM ONE VIEW , 1970 .

[12]  B. Julesz.  Foundations of Cyclopean Perception , 1971 .

[13]  Larry S. Davis,et al.  A survey of edge detection techniques , 1975 .

[14]  Tomaso Poggio,et al.  From Understanding Computation to Understanding Neural Circuitry , 1976 .

[15]  D Marr,et al.  Early processing of visual information. , 1976, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[16]  Robert J. Woodham,et al.  Reflectance map techniques for analyzing surface defects in metal castings , 1978 .

[17]  Tomaso Poggio,et al.  A Theory of Human Stereo Vision , 1977 .

[18]  David Marr,et al.  Representing Visual Information , 1977 .

[19]  Christopher W. Tyler,et al.  Spatial Limitations Of Human Stereoscopic Vision , 1977, Optics & Photonics.

[20]  D. Marr,et al.  Representation and recognition of the spatial organization of three-dimensional shapes , 1978, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[21]  B K Horn,et al.  Calculating the reflectance map. , 1979, Applied optics.

[22]  Shimon Ullman,et al.  Relaxation and constrained optimization by local processes , 1979 .

[23]  T. Poggio,et al.  A computational theory of human stereo vision , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[24]  S. Ullman,et al.  The interpretation of visual motion , 1977 .

[25]  W. Grimson.  Computing shape using a theory of human stereo vision , 1980 .

[26]  Ellen C. Hildreth,et al.  Implementation Of A Theory Of Edge Detection , 1980 .

[27]  H. C. Longuet-Higgins,et al.  The interpretation of a moving retinal image , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[28]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[29]  S. Ullman,et al.  A Model for the Spatio-Temporal Organization of X- and Y-Type Ganglion Cells in the Primate Retina , 1980 .

[30]  D. Marr,et al.  Smallest channel in early human vision. , 1980, Journal of the Optical Society of America.

[31]  Harry G. Barrow,et al.  Interpreting Line Drawings as Three-Dimensional Surfaces , 1980, Artif. Intell..

[32]  John E. W. Mayhew,et al.  Psychophysical and Computational Studies Towards a Theory of Human Stereopsis , 1981, Artif. Intell..

[33]  W. Grimson.  The Implicit Constraints of the Primal Sketch. , 1981 .

[34]  Thomas O. Binford,et al.  Intensity Discontinuity Location to SubPixel Precision , 1981, IJCAI.

[35]  Eric L. W. Grimson,et al.  From Images to Surfaces: A Computational Study of the Human Early Visual System , 1981 .

[36]  W E Grimson,et al.  A computer implementation of a theory of human stereo vision. , 1981, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[37]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[38]  Michael Brady,et al.  Rotationally symmetric operators for surface interpolation , 1983, Comput. Vis. Graph. Image Process..

[39]  W. Eric L. Grimson,et al.  An implementation of a computational theory of visual surface interpolation , 1983, Comput. Vis. Graph. Image Process..