Information-Theoretic Properties of Auditory Sequences Dynamically Influence Expectation and Memory

A basic function of cognition is to detect regularities in sensory input to facilitate the prediction and recognition of future events. It has been proposed that these implicit expectations arise from an internal predictive coding model, based on knowledge acquired through processes such as statistical learning, but it is unclear how different types of statistical information affect listeners' memory for auditory stimuli. We used a combination of behavioral and computational methods to investigate memory for non-linguistic auditory sequences. Participants repeatedly heard tone sequences varying systematically in their information-theoretic properties. Expectedness ratings of tones were collected during three listening sessions, and a recognition memory test was given after each session. Information-theoretic measures of sequential predictability significantly influenced listeners' expectedness ratings, and variations in these properties had a significant impact on memory performance. Predictable sequences yielded increasingly better memory performance with increasing exposure. Computational simulations using a probabilistic model of auditory expectation suggest that listeners dynamically formed a new, and increasingly accurate, implicit cognitive model of the information-theoretic structure of the sequences throughout the experimental session.

[1]  R. Shepard,et al.  Tonal Schemata in the Perception of Music in Bali and in the West , 1984 .

[2]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[3]  Richard C. Atkinson,et al.  Human Memory: A Proposed System and its Control Processes , 1968, Psychology of Learning and Motivation.

[4]  Barbara Tillmann,et al.  Music and Language Perception: Expectations, Structural Integration, and Cognitive Sequencing , 2012, Top. Cogn. Sci..

[5]  James S. Magnuson,et al.  The link between statistical segmentation and word learning in adults , 2008, Cognition.

[6]  C. Krumhansl,et al.  Tracing the dynamic changes in perceived tonal organization in a spatial representation of musical keys. , 1982 .

[7]  M. Pearce,et al.  Predictive uncertainty in auditory sequence processing , 2014, Front. Psychol..

[8]  Morten H. Christiansen,et al.  PSYCHOLOGICAL SCIENCE Research Article Statistical Learning Within and Between Modalities Pitting Abstract Against Stimulus-Specific Representations , 2022 .

[9]  Geraint A. Wiggins,et al.  The Role of Expectation and Probabilistic Learning in Auditory Boundary Perception: A Model Comparison , 2010, Perception.

[10]  James C. Carlsen Some factors which influence melodic expectancy , 1981 .

[11]  Karl J. Friston,et al.  Predictive coding under the free-energy principle , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[12]  Schellenberg Eg Expectancy in melody: tests of the implication-realization model , 1996 .

[13]  Zoltan Dienes,et al.  Can musical transformations be implicitly learned? , 2004, Cogn. Sci..

[14]  J. Saffran,et al.  Musical Learning and Language Development , 2003, Annals of the New York Academy of Sciences.

[15]  W. Dowling,et al.  Tonal strength and melody recognition after long and short delays , 1991, Perception & psychophysics.

[16]  Katherine A. DeLong,et al.  Probabilistic word pre-activation during language comprehension inferred from electrical brain activity , 2005, Nature Neuroscience.

[17]  John Hale,et al.  Uncertainty About the Rest of the Sentence , 2006, Cogn. Sci..

[18]  Joel E. Cohen,et al.  Information theory and music , 2007 .

[19]  Alexa R. Romberg,et al.  Statistical learning and language acquisition. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[20]  Elizabeth K. Johnson,et al.  Statistical learning of tone sequences by human infants and adults , 1999, Cognition.

[21]  L. Knopoff,et al.  Entropy as a Measure of Style: The Influence of Sample Length , 1983 .

[22]  P. Perruchet,et al.  Implicit learning and statistical learning: one phenomenon, two approaches , 2006, Trends in Cognitive Sciences.

[23]  Andrea R. Halpern,et al.  Melody recognition at fast and slow tempos: Effects of age, experience, and familiarity , 2008, Perception & psychophysics.

[24]  Daniel Mirman,et al.  Computational Modeling of Statistical Learning: Effects of Transitional Probability Versus Frequency and Links to Word Learning. , 2010, Infancy : the official journal of the International Society on Infant Studies.

[25]  M. Pearce,et al.  Electrophysiological correlates of melodic processing in congenital amusia , 2013, Neuropsychologia.

[26]  Jonathan P. How,et al.  Estimation of non-stationary Markov Chain transition models , 2008, 2008 47th IEEE Conference on Decision and Control.

[27]  Alejandrina Cristià,et al.  Effects of the distribution of acoustic cues on infants' perception of sibilants , 2011, J. Phonetics.

[28]  M. Tervaniemi,et al.  Development of a memory trace for a complex sound in the human brain. , 1993, Neuroreport.

[29]  C. Krumhansl,et al.  Tonal hierarchies in the music of north India. , 1984, Journal of experimental psychology. General.

[30]  Martin Rohrmeier,et al.  Implicit Learning and Acquisition of Music , 2012, Top. Cogn. Sci..

[31]  E. Schellenberg,et al.  Expectancy in melody: tests of the implication-realization model , 1996, Cognition.

[32]  A. Clark Whatever next? Predictive brains, situated agents, and the future of cognitive science. , 2013, The Behavioral and brain sciences.

[33]  Ian H. Witten,et al.  Multiple viewpoint systems for music prediction , 1995 .

[34]  J. Saffran,et al.  Absolute pitch in infant auditory learning: evidence for developmental reorganization. , 2001, Developmental psychology.

[35]  L. Cuddy,et al.  Responsiveness of Western adults to pitch-distributional information in melodic sequences , 1995, Psychological research.

[36]  A. Baddeley,et al.  When long-term learning depends on short-term storage , 1988 .

[37]  I. Cross,et al.  Incidental and online learning of melodic structure , 2011, Consciousness and Cognition.

[38]  Roy D. Patterson,et al.  Predictive Coding and Pitch Processing in the Auditory Cortex , 2011, Journal of Cognitive Neuroscience.

[39]  Andrea R. Halpern,et al.  Recognition of familiar and unfamiliar melodies in normal aging and Alzheimer’s disease , 1995, Memory & cognition.

[40]  Geoffrey E. Hinton Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[41]  M. Bar The proactive brain: using analogies and associations to generate predictions , 2007, Trends in Cognitive Sciences.

[42]  Ran El-Yaniv,et al.  On Prediction Using Variable Order Markov Models , 2004, J. Artif. Intell. Res..

[43]  Mark D. Plumbley,et al.  Information dynamics: patterns of expectation and surprise in the perception of music , 2009, Connect. Sci..

[44]  Marcus T. Pearce,et al.  The construction and evaluation of statistical models of melodic structure in music perception and composition , 2005 .

[45]  Daniel Müllensiefen,et al.  Effects of timbre and tempo change on memory for music , 2008, Quarterly journal of experimental psychology.

[46]  D. Wolpert,et al.  Motor prediction , 2001, Current Biology.

[47]  Nick Chater,et al.  Distributional Information: A Powerful Cue for Acquiring Syntactic Categories , 1998, Cogn. Sci..

[48]  Charles Ames,et al.  The Markov Process as a Compositional Model: A Survey and Tutorial , 2017 .

[49]  Andrea R. Halpern,et al.  The Role of Features and Context in Recognition of Novel Melodies , 2014 .

[50]  F. Bartlett,et al.  Remembering: A Study in Experimental and Social Psychology , 1932 .

[51]  John Shawe-Taylor,et al.  Neural prediction of higher-order auditory sequence statistics , 2011, NeuroImage.

[52]  Giorgio Vallortigara,et al.  Unsupervised statistical learning in newly hatched chicks , 2016, Current Biology.

[53]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[54]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[55]  E. Schellenberg,et al.  Simplifying the Implication-Realization Model of Melodic Expectancy , 1997 .

[56]  M. Pearce,et al.  Tracking of pitch probabilities in congenital amusia , 2012, Neuropsychologia.

[57]  I. Dean,et al.  Neural population coding of sound level adapts to stimulus statistics , 2005, Nature Neuroscience.

[58]  Matthias Abend Cognitive Foundations Of Musical Pitch , 2016 .

[59]  C. Krumhansl,et al.  Melodic Expectation in Finnish Spiritual Folk Hymns: Convergence of Statistical, Behavioral, and Computational Approaches , 1999 .

[60]  M. Goldsmith,et al.  Statistical Learning by 8-Month-Old Infants , 1996 .

[61]  William Hutchinson,et al.  Information Theory for Musical Continua , 1981 .

[62]  T. Eerola Data-driven influences on melodic expectancy : Continuations in North Sami yoiks rated by South African traditional healers , 2004 .

[63]  Geoffrey E. Hinton,et al.  The Helmholtz Machine , 1995, Neural Computation.

[64]  D. Levitin,et al.  Current Advances in the Cognitive Neuroscience of Music , 2009, Annals of the New York Academy of Sciences.

[65]  Axel Cleeremans,et al.  Implicit learning: news from the front , 1998, Trends in Cognitive Sciences.

[66]  R. Aslin,et al.  Statistical learning in a serial reaction time task: access to separable statistical cues by individual learners. , 2001, Journal of experimental psychology. General.

[67]  B. Tillmann,et al.  Implicit learning of tonality: a self-organizing approach. , 2000, Psychological review.

[68]  M. Pearce,et al.  Sweet Anticipation : Music and the Psychology of Expectation , 2007 .

[69]  J. Saffran,et al.  Changing the tune: the structure of the input affects infants' use of absolute and relative pitch. , 2005, Developmental science.

[70]  Mark D. Plumbley,et al.  A measure of statistical complexity based on predictive information , 2010, ArXiv.

[71]  Jim M. Monti,et al.  Expectation and Surprise Determine Neural Population Responses in the Ventral Visual Stream , 2010, The Journal of Neuroscience.

[72]  W. L. Windsor Music and Probability , 2009 .

[73]  Morten H. Christiansen,et al.  Modality-constrained statistical learning of tactile, visual, and auditory sequences. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[74]  Geraint A. Wiggins,et al.  Auditory Expectation: The Information Dynamics of Music Perception and Cognition , 2012, Top. Cogn. Sci..

[75]  Mark D. Plumbley,et al.  predictive information , multi-information , and binding information , 2010 .

[76]  María Herrojo Ruiz,et al.  Unsupervised statistical learning underpins computational, behavioural, and neural manifestations of musical expectation , 2010, NeuroImage.

[77]  C. Krumhansl,et al.  Cross-cultural music cognition: cognitive methodology applied to North Sami yoiks , 2000, Cognition.

[78]  Leonard B. Meyer Emotion and Meaning in Music , 1957 .

[79]  R. Shepard Geometrical approximations to the structure of musical pitch. , 1982, Psychological review.

[80]  J. Saffran,et al.  Dog is a dog is a dog: Infant rule learning is not specific to language , 2007, Cognition.

[81]  W. Stolz Information Theory and Esthetic Perception. , 1967 .

[82]  J. Youngblood Style as Information , 1958 .

[83]  Scott P. Johnson,et al.  Visual statistical learning in infancy: evidence for a domain general learning mechanism , 2002, Cognition.

[84]  D J Mewhort,et al.  Perception of structure in short melodic sequences. , 1981, Journal of experimental psychology. Human perception and performance.

[85]  Geraint A. Wiggins,et al.  EXPECTATION IN MELODY: THE INFLUENCE OF CONTEXT AND LEARNING , 2006 .

[86]  Geraint A. Wiggins,et al.  Probabilistic models of expectation violation predict psychophysiological emotional responses to live concert music , 2013, Cognitive, Affective, & Behavioral Neuroscience.

[87]  R. Gregory Perceptions as hypotheses. , 1980, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[88]  Gordon D. A. Brown,et al.  Memory for familiar and unfamiliar words: Evidence for a long-term memory contribution to short-term memory span , 1991 .

[89]  Barbara Tillmann,et al.  Effects of Global and Local Contexts on Harmonic Expectancy , 1998 .

[90]  J. Bharucha Music Cognition and Perceptual Facilitation: A Connectionist Framework , 1987 .

[91]  D. Wessel,et al.  Learning and Liking an Artificial Musical System: Effects of Set Size and Repeated Exposure , 2008, Musicae scientiae : the journal of the European Society for the Cognitive Sciences of Music.

[92]  E. Newport,et al.  WORD SEGMENTATION : THE ROLE OF DISTRIBUTIONAL CUES , 1996 .

[93]  J. Bartlett,et al.  Memory for Melodies , 2010 .

[94]  J. Saffran Statistical Language Learning , 2003 .

[95]  R. Aslin,et al.  Statistical learning of higher-order temporal structure from visual shape sequences. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[96]  Marcus T. Pearce,et al.  An Information-Theoretic Account of Musical Expectation and Memory , 2013, CogSci.

[97]  M. Brent Speech segmentation and word discovery: a computational perspective , 1999, Trends in Cognitive Sciences.

[98]  Eugene Narmour,et al.  The Analysis and Cognition of Basic Melodic Structures: The Implication-Realization Model , 1990 .

[99]  Suzanne Bunton,et al.  Semantically Motivated Improvements for PPM Variants , 1997, Comput. J..

[100]  Sarah C. Creel,et al.  Distant melodies: statistical learning of nonadjacent dependencies in tone sequences. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[101]  Carla L. Hudson Kam,et al.  Humans Rapidly Learn Grammatical Structure in a New Musical Scale. , 2010, Music perception.

[102]  K. A. Ericsson,et al.  Long-term working memory. , 1995, Psychological review.