Derivatives and inverse of a linear-nonlinear multi-layer spatial vision model

Linear-nonlinear transforms are interesting in vision science because they are key in modeling a number of perceptual experiences such as color, motion or spatial texture. Here we first show that a number of issues in vision may be addressed through an analytic expression of the Jacobian of these linear-nonlinear transforms. The particular model analyzed afterwards (an extension of [Malo & Simoncelli SPIE 2015]) is illustrative because it consists of a cascade of standard linear-nonlinear modules. Each module roughly corresponds to a known psychophysical mechanism: (1) linear spectral integration and nonlinear brightness-from-luminance computation, (2) linear pooling of local brightness and nonlinear normalization for local contrast computation, (3) linear frequency selectivity and nonlinear normalization for spatial contrast masking, and (4) linear wavelet-like decomposition and nonlinear normalization for frequency-dependent masking. Beyond being the appropriate technical report with the missing details in [Malo & Simoncelli SPIE 2015], the interest of the presented analytic results and numerical methods transcend the particular model because of the ubiquity of the linear-nonlinear structure. Part of this material was presented at MODVIS 2016 (see slides of the conference talk in the appendix at the end of this document).

[1]  P. Mahalanobis On the generalized distance in statistics , 1936 .

[2]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[3]  J. M. Foley,et al.  Human luminance pattern-vision mechanisms: masking experiments require a new model. , 1994, Journal of the Optical Society of America. A, Optics, image science, and vision.

[4]  Xin Wang,et al.  Statistical Wiring of Thalamic Receptive Fields Optimizes Spatial Sampling of the Retinal Image , 2014, Neuron.

[5]  Valero Laparra,et al.  Iterative Gaussianization: From ICA to Random Rotations , 2011, IEEE Transactions on Neural Networks.

[6]  Edward H. Adelson,et al.  The Laplacian Pyramid as a Compact Image Code , 1983, IEEE Trans. Commun..

[7]  Valero Laparra,et al.  Nonlinearities and Adaptation of Color Vision from Sequential Principal Curves Analysis , 2016, Neural Computation.

[8]  J A Solomon,et al.  Model of visual contrast gain control and pattern masking. , 1997, Journal of the Optical Society of America. A, Optics, image science, and vision.

[9]  Eero P. Simoncelli,et al.  Maximum differentiation (MAD) competition: a methodology for comparing computational models of perceptual quantities. , 2008, Journal of vision.

[10]  D. Burr,et al.  Motion psychophysics: 1985–2010 , 2011, Vision Research.

[11]  Eero P. Simoncelli,et al.  Natural signal statistics and sensory gain control , 2001, Nature Neuroscience.

[12]  Andrew B. Watson,et al.  The cortex transform: rapid computation of simulated neural images , 1987 .

[13]  Valero Laparra,et al.  Visual aftereffects and sensory nonlinearities from a single statistical framework , 2015, Front. Hum. Neurosci..

[14]  W D Wright,et al.  Color Science, Concepts and Methods. Quantitative Data and Formulas , 1967 .

[15]  Valero Laparra,et al.  Psychophysically Tuned Divisive Normalization Approximately Factorizes the PDF of Natural Images , 2010, Neural Computation.

[16]  Valero Laparra,et al.  Divisive normalization image quality metric revisited. , 2010, Journal of the Optical Society of America. A, Optics, image science, and vision.

[17]  Bernd Girod,et al.  Subband Image Coding , 1996 .

[18]  J. Malo,et al.  V1 non-linear properties emerge from local-to-global non-linear ICA , 2006, Network.

[19]  Jonathan Winawer,et al.  A Two-Stage Cascade Model of BOLD Responses in Human Visual Cortex , 2013, PLoS Comput. Biol..

[20]  오승준 [서평]「Digital Video Processing」 , 1996 .

[21]  H Barlow,et al.  Redundancy reduction revisited , 2001, Network.

[22]  William T. Freeman,et al.  Presented at: 2nd Annual IEEE International Conference on Image , 1995 .

[23]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[24]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[25]  David Kane,et al.  The Maximum Differentiation competition depends on the Viewing Conditions , 2016 .

[26]  Patrick C. Teo,et al.  Perceptual image distortion , 1994, Proceedings of 1st International Conference on Image Processing.

[27]  Jesús Malo,et al.  Linear transform for simultaneous diagonalization of covariance and perceptual metric matrix in image coding , 2003, Pattern Recognit..

[28]  S. Laughlin,et al.  Matching Coding to Scenes to Enhance Efficiency , 1983 .

[29]  L. Rudin,et al.  Nonlinear total variation based noise removal algorithms , 1992 .

[30]  James M. Hillis,et al.  Do common mechanisms of adaptation mediate color discrimination and appearance? Contrast adaptation. , 2007, Journal of the Optical Society of America. A, Optics, image science, and vision.

[31]  David J. Field,et al.  Emergence of simple-cell receptive field properties by learning a sparse code for natural images , 1996, Nature.

[32]  Seungjin Choi,et al.  Independent Component Analysis , 2009, Handbook of Natural Computing.

[33]  Eero P. Simoncelli,et al.  A model of neuronal responses in visual area MT , 1998, Vision Research.

[34]  Frank Tong,et al.  Foundations of Vision , 2018 .

[35]  G. Wyszecki,et al.  Color Science Concepts and Methods , 1982 .

[36]  Mahdi Nezamabadi,et al.  Color Appearance Models , 2014, J. Electronic Imaging.

[37]  S Marcelja,et al.  Mathematical description of the responses of simple cortical cells. , 1980, Journal of the Optical Society of America.

[38]  Jesús Malo,et al.  Video quality measures based on the standard spatial observer , 2002, Proceedings. International Conference on Image Processing.

[39]  J. M. Foley,et al.  Contrast masking in human vision. , 1980, Journal of the Optical Society of America.

[40]  Eero P. Simoncelli,et al.  Geometrical and statistical properties of vision models obtained via maximum differentiation , 2015, Electronic Imaging.

[41]  T. Minka Old and New Matrix Algebra Useful for Statistics , 2000 .

[42]  Eero P. Simoncelli,et al.  Nonlinear image representation for efficient perceptual coding , 2006, IEEE Transactions on Image Processing.

[43]  A B Watson,et al.  Efficiency of a model human image code. , 1987, Journal of the Optical Society of America. A, Optics and image science.

[44]  M. Studený,et al.  The Multiinformation Function as a Tool for Measuring Stochastic Dependence , 1998, Learning in Graphical Models.

[45]  J. Robson,et al.  Application of fourier analysis to the visibility of gratings , 1968, The Journal of physiology.

[46]  Donald I. A. MacLeod,et al.  The pleistochrome: optimal opponent codes for natural colours , 2003 .

[47]  Jean-François Cardoso,et al.  Dependence, Correlation and Gaussianity in Independent Component Analysis , 2003, J. Mach. Learn. Res..

[48]  Eero P. Simoncelli,et al.  Nonlinear Extraction of Independent Components of Natural Images Using Radial Gaussianization , 2009, Neural Computation.

[49]  B. Dubrovin,et al.  Modern geometry--methods and applications , 1984 .