Visual Motion and the Perception of Surface Material

Many critical perceptual judgments, from telling whether fruit is ripe to determining whether the ground is slippery, involve estimating the material properties of surfaces. Very little is known about how the brain recognizes materials, even though the problem is likely as important for survival as navigating or recognizing objects. Though previous research has focused nearly exclusively on the properties of static images, recent evidence suggests that motion may affect the appearance of surface material. However, what kind of information motion conveys and how this information may be used by the brain is still unknown. Here, we identify three motion cues that the brain could rely on to distinguish between matte and shiny surfaces. We show that these motion measurements can override static cues, leading to dramatic changes in perceived material depending on the image motion characteristics. A classifier algorithm based on these cues correctly predicts both successes and some striking failures of human material perception. Together these results reveal a previously unknown use for optic flow in the perception of surface material properties.

[1]  Yuli D. Chashechkin,et al.  Transportation of a dye in upstream and downstream wakes of the cylinder in continuously stratified liquid , 2007, J. Vis..

[2]  A. Torralba,et al.  Specular reflections and the perception of shape. , 2004, Journal of vision.

[3]  J. Koenderink,et al.  Photometric Invariants Related to Solid Shape , 1980 .

[4]  Richard Szeliski,et al.  On the Motion and Appearance of Specularities in Image Sequences , 2002, ECCV.

[5]  M. Graziano,et al.  Tuning of MST neurons to spiral motions , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[6]  Vebjørn Ekroll,et al.  Disparity, motion, and color information improve gloss constancy performance. , 2010, Journal of vision.

[7]  Marc M. Van Hulle,et al.  A phase-based approach to the estimation of the optical flow field using spatial filtering , 2002, IEEE Trans. Neural Networks.

[8]  Paul R. Schrater,et al.  Rapid classification of specular and diffuse reflection from image velocities , 2011, Pattern Recognit..

[9]  Rajiv Gupta,et al.  Stereo from uncalibrated cameras , 1992, Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[10]  S. Nishida,et al.  Use of image-based information in judgments of surface-reflectance properties. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[11]  Yuichi Sakano,et al.  Effects of self-motion on gloss perception , 2008 .

[12]  Wojciech Matusik,et al.  A data-driven reflectance model , 2003, ACM Trans. Graph..

[13]  Thomas Rösgen,et al.  Optical density visualization and abel reconstruction of vortex rings using background-oriented schlieren , 2007, J. Vis..

[14]  M. Landy,et al.  How direction of illumination affects visually perceived surface roughness. , 2006, Journal of vision.

[15]  J Zeil,et al.  Signals from 'crabworld': cuticular reflections in a fiddler crab colony. , 2001, The Journal of experimental biology.

[16]  Heinrich H. Bülthoff,et al.  Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, APGV 2005, A Coruña, Spain, August 26-28, 2005 , 2005, APGV.

[17]  J. Koenderink,et al.  Detection of divergence in optical flow fields , 1996 .

[18]  E. Adelson,et al.  Image statistics and the perception of surface qualities , 2007, Nature.

[19]  Alexander Grunewald,et al.  Neural Correlates of Structure-from-Motion Perception in Macaque V1 and MT , 2002, The Journal of Neuroscience.

[20]  Phillip J. Marlow,et al.  The perception of gloss depends on highlight congruence with surface shading. , 2011, Journal of vision.

[21]  D. Ingle,et al.  Brain mechanisms and spatial vision , 1985 .

[22]  James T. Todd,et al.  The Analysis of Three-Dimensional Structure from Moving Images , 1985 .

[23]  Barton L Anderson,et al.  The role of brightness and orientation congruence in the perception of surface gloss. , 2011, Journal of vision.

[24]  Jonathan S. Cant,et al.  fMR-adaptation reveals separate processing regions for the perception of form and texture in the human ventral stream , 2008, Experimental Brain Research.

[25]  Shree K. Nayar,et al.  A Theory of Specular Surface Geometry , 2004, International Journal of Computer Vision.

[26]  K. Tanaka,et al.  Analysis of motion of the visual field by direction, expansion/contraction, and rotation cells clustered in the dorsal part of the medial superior temporal area of the macaque monkey. , 1989, Journal of neurophysiology.

[27]  Qasim Zaidi,et al.  Visual inferences of material changes: color as clue and distraction. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[28]  Robert P. W. Duin,et al.  A Matlab Toolbox for Pattern Recognition , 2004 .

[29]  H. Bülthoff,et al.  Does the brain know the physics of specular reflection? , 1990, Nature.

[30]  Y. Nakayama,et al.  Memorial message for the 10th anniversary of journal of visualization , 2007, J. Vis..

[31]  G. W. Larson,et al.  Rendering with radiance - the art and science of lighting visualization , 2004, Morgan Kaufmann series in computer graphics and geometric modeling.

[32]  R. Schwind Spectral regions in which aquatic insects see reflected polarized light , 1995, Journal of Comparative Physiology A.

[33]  Daniel Kersten,et al.  Distinguishing shiny from matte , 2010 .

[34]  Jan J. Koenderink,et al.  Space, Form and Optical Deformations , 1985 .

[35]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[36]  Keiji Tanaka,et al.  Integration of direction signals of image motion in the superior temporal sulcus of the macaque monkey , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[37]  Robert P.W. Duin,et al.  PRTools3: A Matlab Toolbox for Pattern Recognition , 2000 .

[38]  Huseyin Boyaci,et al.  Estimating the glossiness transfer function induced by illumination change and testing its transitivity. , 2010, Journal of vision.

[39]  Edward H. Adelson,et al.  Estimating surface reflectance properties from images under unknown illumination , 2001, IS&T/SPIE Electronic Imaging.

[40]  Barton L Anderson,et al.  Image statistics do not explain the perception of gloss and lightness. , 2009, Journal of vision.

[41]  Nao Ninomiya,et al.  The 10th anniversary of journal of visualization , 2007, J. Vis..

[42]  Barton L Anderson,et al.  Image statistics and the perception of surface gloss and lightness. , 2010, Journal of vision.

[43]  Qasim Zaidi,et al.  Discerning nonrigid 3D shapes from motion cues , 2011, Proceedings of the National Academy of Sciences.

[44]  D. Janzen,et al.  A tropical horde of counterfeit predator eyes , 2010, Proceedings of the National Academy of Sciences.

[45]  James R. Schott,et al.  Principles of Multivariate Analysis: A User's Perspective , 2002 .

[46]  PAUL D. SAMPSON,et al.  Fitting conic sections to "very scattered" data: An iterative refinement of the bookstein algorithm , 1982, Comput. Graph. Image Process..

[47]  Sylvia C. Pont,et al.  A comparison of material and illumination discrimination performance for real rough, real smooth and computer generated smooth spheres , 2005, APGV '05.

[48]  Maria Olkkonen,et al.  Perceived glossiness and lightness under real-world illumination. , 2010, Journal of vision.

[49]  Bruce G. Cumming,et al.  RECOGNITION AND PERCEPTUAL USE OF SPECULAR REFLECTIONS , 1991 .

[50]  Andrew Blake,et al.  Specular Stereo , 1985, IJCAI.

[51]  Philip Dutré,et al.  The influence of shape on the perception of material reflectance , 2007, ACM Trans. Graph..

[52]  Edward H. Adelson,et al.  On seeing stuff: the perception of materials by humans and machines , 2001, IS&T/SPIE Electronic Imaging.