The Morton-Massaro law of information integration: implications for models of perception.

Information integration may be studied by analyzing the effect of 2 or more sources (e.g., auditory and visual) on participants' responses. Experiments show that ratios of response probabilities often factorize into components selectively influenced by only 1 source (e.g., 1 component affected by the acoustic source and another 1 affected by the visual source). This is called the Morton-Massaro law (MML). This article identifies conditions in which the law is optimal and notes that it reflects an implicit assumption about the statistics of the environment. Adherence to the MML can be used to assess whether the assumption is being made, and analyses of natural stimuli can be used to determine whether the assumption is reasonable. Feed-forward and interactive models subject to a channel separability constraint are consistent with the law.

[1]  R. Shiffrin,et al.  A retrieval model for both recognition and recall. , 1984, Psychological review.

[2]  John R. Anderson.  The Adaptive Character of Thought , 1990 .

[3]  William H. Press,et al.  Numerical Recipes in FORTRAN - The Art of Scientific Computing, 2nd Edition , 1987 .

[4]  Peter Dayan,et al.  Exploration bonuses and dual control , 1996 .

[5]  D. Massaro.  Speech Perception By Ear and Eye: A Paradigm for Psychological Inquiry , 1989 .

[6]  James T. Townsend,et al.  The Stochastic Modeling of Elementary Psychological Processes , 1983 .

[7]  B. Efron.  The jackknife, the bootstrap, and other resampling plans , 1987 .

[8]  Ioannis Karatzas,et al.  Brownian Motion and Stochastic Calculus , 1987 .

[9]  H. Kucera,et al.  Computational analysis of present-day American English , 1967 .

[10]  M A Nicolelis,et al.  Nonlinear processing of tactile information in the thalamocortical loop. , 1997, Journal of neurophysiology.

[11]  D. Stork,et al.  Speechreading by Man and Machine: Models, Systems, and Applications , 1996 .

[12]  Saul Sternberg,et al.  The discovery of processing stages: Extensions of Donders' method , 1969 .

[13]  James J. Clark,et al.  Data Fusion for Sensory Information Processing Systems , 1990 .

[14]  D. Massaro,et al.  Integration versus interactive activation: The joint influence of stimulus and context in perception , 1991, Cognitive Psychology.

[15]  Javier R. Movellan,et al.  Learning Continuous Probability Distributions with Symmetric Diffusion Networks , 1993, Cogn. Sci..

[16]  Allen Newell,et al.  The Knowledge Level , 1989, Artif. Intell..

[17]  James L. McClelland.  Stochastic Interactive Activation and the Effect of Context on Perception , 1989 .

[18]  Elie Bienenstock,et al.  Neural Networks and the Bias/Variance Dilemma , 1992, Neural Computation.

[19]  D. Massaro,et al.  Phonological context in speech perception , 1983, Perception & psychophysics.

[20]  Douglas Vickers,et al.  Human Information Processing: Measures, Mechanisms, and Models , 1990 .

[21]  Eugene Galanter,et al.  Handbook of mathematical psychology: I. , 1963 .

[22]  G S Dell,et al.  A spreading-activation theory of retrieval in sentence production. , 1986, Psychological review.

[23]  Hh Bülthoff,et al.  A Bayesian framework for the integration of visual modules , 1996 .

[24]  J T Townsend,et al.  Perceptual sampling of orthogonal straight line features , 1981, Psychological research.

[25]  D. J. Felleman,et al.  Distributed hierarchical processing in the primate cerebral cortex. , 1991, Cerebral cortex.

[26]  J. Allman,et al.  Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. , 1985, Annual review of neuroscience.

[27]  N. Perrin,et al.  Varieties of perceptual independence. , 1986, Psychological review.

[28]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[29]  Juergen Luettin,et al.  Statistical LIP modelling for visual speech recognition , 1996, 1996 8th European Signal Processing Conference (EUSIPCO 1996).

[30]  Richard M. Golden.  A Developmental Neural Model of Visual Word Perception , 1986 .

[31]  D. Rumelhart,et al.  Process of recognizing tachistoscopically presented words. , 1974, Psychological review.

[32]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[33]  Ashby,et al.  A Stochastic Version of General Recognition Theory. , 2000, Journal of mathematical psychology.

[34]  Gregory J. Wolff,et al.  Preprocessing video images for neural learning of lipreading , 1994, Other Conferences.

[35]  P. Kloeden,et al.  Numerical Solution of Stochastic Differential Equations , 1992 .

[36]  Biing-Hwang Juang,et al.  Fundamentals of speech recognition , 1993, Prentice Hall signal processing series.

[37]  Xiangen Hu,et al.  A measurement-theoretic analysis of the fuzzy logic model of perception. , 1995 .

[38]  W. Estes.  The Locus of Inferential and Perceptual Processes in Letter Identification. , 1975 .

[39]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[40]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[41]  Javier R. Movellan,et al.  A Learning Theorem for Networks at Detailed Stochastic Equilibrium , 1998, Neural Computation.

[42]  Daniel McFadden,et al.  Modelling the Choice of Residential Location , 1977 .

[43]  D. Massaro,et al.  Models of integration given multiple sources of information. , 1990, Psychological review.

[44]  R. Sperry.  A modified concept of consciousness. , 1969, Psychological review.

[45]  J. Laurie Snell,et al.  Topics in Contemporary Probability and Its Applications , 1995 .

[46]  J J Hopfield,et al.  Neurons with graded response have collective computational properties like those of two-state neurons. , 1984, Proceedings of the National Academy of Sciences of the United States of America.

[47]  W. R. Garner,et al.  Perceptual independence: definitions, models, and experimental paradigms. , 1969, Psychological bulletin.

[48]  R W Schvaneveldt,et al.  An activation--verification model for letter and word recognition: the word-superiority effect. , 1982, Psychological review.

[49]  Geoffrey E. Hinton,et al.  Schemata and Sequential Thought Processes in PDP Models , 1986 .

[50]  B. Stein,et al.  The Merging of the Senses , 1993 .

[51]  Geoffrey E. Hinton,et al.  Lesioning an attractor network: investigations of acquired dyslexia , 1991 .

[52]  B. Øksendal.  Stochastic Differential Equations , 1985 .

[53]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[54]  R. Shepard.  Attention and the metric structure of the stimulus space. , 1964 .

[55]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[56]  William H. Press,et al.  Numerical recipes in C. The art of scientific computing , 1987 .

[57]  D. Massaro,et al.  Integration of featural information in speech perception. , 1978, Psychological review.

[58]  Dominic W. Massaro,et al.  Experimental Psychology: An Information Processing Approach , 1989 .

[59]  P. Marler,et al.  Communication Goes Multimodal , 1999, Science.

[60]  Jonathan Grainger,et al.  A dual read-out model of word context effects in letter perception: Further investigations of the word superiority effect. , 1994 .

[61]  D. Massaro,et al.  Phonological constraints in speech perception , 1980 .

[62]  Paul Mineiro,et al.  Learning Path Distributions Using Nonequilibrium Diffusion Networks , 1997, NIPS.

[63]  D. Massaro,et al.  Perceiving Talking Faces , 1995 .

[64]  Javier R. Movellan,et al.  Channel Separability in the Audio-Visual Integration of Speech: A Bayesian Approach , 1996 .

[65]  J. Yellott.  The relationship between Luce's Choice Axiom, Thurstone's Theory of Comparative Judgment, and the double exponential distribution , 1977 .

[66]  M. Arbib,et al.  Conceptual models of neural organization. , 1974, Neurosciences Research Program bulletin.

[67]  John K. Kruschke,et al.  Dimensional Relevance Shifts in Category Learning , 1996, Connect. Sci..

[68]  James L. McClelland.  Stochastic interactive processes and the effect of context on perception , 1991, Cognitive Psychology.

[69]  D. Mumford,et al.  The role of the primary visual cortex in higher level vision , 1998, Vision Research.

[70]  D. Massaro.  Testing between the TRACE model and the fuzzy logical model of speech perception , 1989, Cognitive Psychology.

[71]  A. Samuel.  Does lexical information influence the perceptual restoration of phonemes , 1996 .

[72]  Javier R. Movellan,et al.  Dynamic Features for Visual Speechreading: A Systematic Comparison , 1996, NIPS.

[73]  John J. L. Morton,et al.  Interaction of information in word recognition. , 1969 .

[74]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[75]  James L. McClelland,et al.  An interactive activation model of context effects in letter perception: I. An account of basic findings. , 1981 .

[76]  D L Weber,et al.  Detection and recognition of pure tones in noise. , 1977, The Journal of the Acoustical Society of America.

[77]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[78]  Arthur G. Samuel,et al.  Phonemic Restoration: Insights From a New Methodology , 1981 .

[79]  David G. Stork,et al.  Visionary Speech: Looking Ahead to Practical Speechreading Systems , 1996 .

[80]  R. H. Myers.  Classical and modern regression with applications , 1986 .

[81]  D. Hubel,et al.  Receptive fields and functional architecture of monkey striate cortex , 1968, The Journal of physiology.

[82]  A. Meltzoff,et al.  The bimodal perception of speech in infancy. , 1982, Science.

[83]  R. H. Phaf,et al.  SLAM: A connectionist model for attention in visual selection tasks , 1990, Cognitive Psychology.

[84]  James L. McClelland.  Toward a theory of information processing in graded, random, and interactive networks , 1993 .

[85]  G. Keppel,et al.  Design and Analysis: A Researcher's Handbook , 1976 .

[86]  Victor A. F. Lamme.  The neurophysiology of figure-ground segregation in primary visual cortex , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[87]  Dominic W. Massaro,et al.  Understanding variability in binary and continuous choice , 1998 .

[88]  A. Jacobs,et al.  Models of visual word recognition: Sampling the state of the art. , 1994 .

[89]  J Grainger,et al.  Orthographic processing in visual word recognition: a multiple read-out model. , 1996, Psychological review.

[90]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[91]  Bruno H. Repp,et al.  Exploring the “McGurk effect” , 1983 .

[92]  R. Hari,et al.  Seeing speech: visual information from lip movements modifies activity in the human auditory cortex , 1991, Neuroscience Letters.

[93]  D. Santiago Ramon Y Cajal,et al.  Nuevo Concepto de la Histologia de los Centros Nerviosos , 1893, The Indian Medical Gazette.

[94]  R. Duncan Luce,et al.  Individual Choice Behavior , 1959 .