The butterfly effect: amplification of local changes along the temporal processing hierarchy

Changing just a few words in a story can induce a substantial change in the overall narrative. How does the brain accumulate and process local and sparse changes, creating a unique situation model of the story, over the course of a real-life narrative? Recently, we mapped a hierarchy of processing timescales in the brain: from early sensory areas that integrate information over 10s-100s ms, to high-order areas that integrate information over many seconds to minutes. Based on this hierarchy, we hypothesize that early sensory areas would be sensitive to local changes in word use, but that there will be increasingly divergent neural responses along the processing hierarchy as higher-order areas accumulate and amplify these local changes. To test this hypothesis, we created two structurally related but interpretively distinct narratives by changing some individual words. We found that the neural response distance between the stories was amplified as story information is transferred from low-level regions (e.g. early auditory cortex) to high-level regions (e.g precuneus and prefrontal cortex) and that the neural difference between stories is highly correlated with an area’s ability to integrate information over time. Our results suggest a neural mechanism by which two similar situations become easy to distinguish.

[1]  D. Poeppel,et al.  Cortical Tracking of Hierarchical Linguistic Structures in Connected Speech , 2015, Nature Neuroscience.

[2]  Jussi Tohka,et al.  How Many Is Enough? Effect of Sample Size in Inter-Subject Correlation Analysis of fMRI , 2016, Comput. Intell. Neurosci..

[3]  B. Wandell,et al.  Compressive spatial summation in human visual cortex. , 2013, Journal of neurophysiology.

[4]  Jonathan W. Pillow,et al.  Discovering Event Structure in Continuous Narrative Perception and Memory , 2016, Neuron.

[5]  D. Poeppel,et al.  The cortical organization of speech processing , 2007, Nature Reviews Neuroscience.

[6]  Michael F. Bonner,et al.  Converging Evidence for the Neuroanatomic Basis of Combinatorial Semantics in the Angular Gyrus , 2015, The Journal of Neuroscience.

[7]  Colin Humphries,et al.  Hierarchical organization of speech perception in human auditory cortex , 2014, Front. Neurosci..

[8]  Nancy Kanwisher,et al.  Neural correlate of the construction of sentence meaning , 2016, Proceedings of the National Academy of Sciences.

[9]  Maureen Dennis,et al.  Poor Reading Comprehension Despite Fast Word Decoding in Children with Hydrocephalus , 2001, Brain and Language.

[10]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[11]  C. Honey,et al.  Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story , 2011, The Journal of Neuroscience.

[12]  Karl J. Friston,et al.  A Hierarchy of Time-Scales and the Brain , 2008, PLoS Comput. Biol..

[13]  K. Grill-Spector,et al.  The human visual cortex. , 2004, Annual review of neuroscience.

[14]  M. Grossman,et al.  The organization of narrative discourse in Lewy body spectrum disorder , 2011, Brain and Language.

[15]  D. Heeger,et al.  Slow Cortical Dynamics and the Accumulation of Information over Long Timescales , 2012, Neuron.

[16]  David J. Freedman,et al.  A hierarchy of intrinsic timescales across primate cortex , 2014, Nature Neuroscience.

[17]  Elizabeth Jefferies,et al.  Situating the default-mode network along a principal gradient of macroscale cortical organization , 2016, Proceedings of the National Academy of Sciences.

[18]  E. T. Possing,et al.  Human temporal lobe activation by speech and nonspeech sounds. , 2000, Cerebral cortex.

[19]  Jessica S. Arsenault,et al.  Distributed Neural Representations of Phonological Features during Speech Perception , 2015, The Journal of Neuroscience.

[20]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[21]  Talma Hendler,et al.  Vase or face? A neural correlate of shape-selective grouping processes in the human brain , 2001, NeuroImage.

[22]  Jeffrey M. Zacks,et al.  Human brain activity time-locked to perceptual event boundaries , 2001, Nature Neuroscience.

[23]  Keith A. Johnson,et al.  Stepwise Connectivity of the Modal Cortex Reveals the Multimodal Organization of the Human Brain , 2012, The Journal of Neuroscience.

[24]  M Dennis,et al.  Reading in children and adolescents after early onset hydrocephalus and in normally developing age peers: phonological analysis, word recognition, word comprehension, and passage comprehension skill. , 1992, Journal of pediatric psychology.

[25]  Brian A. Wandell,et al.  Population receptive field estimates in human visual cortex , 2008, NeuroImage.

[26]  A. T. Smith,et al.  Estimating receptive field size from fMRI data in human striate and extrastriate visual cortex. , 2001, Cerebral cortex.

[27]  Christopher J. Honey,et al.  Contextual Alignment of Cognitive and Neural Dynamics , 2015, Journal of Cognitive Neuroscience.

[28]  C. Gaál.  [About hierarchy]. , 2003, Orvosi Hetilap.

[29]  D. Heeger,et al.  A Hierarchy of Temporal Receptive Windows in Human Cortex , 2008, The Journal of Neuroscience.

[30]  M. Torrens.  Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[31]  Alex R. Wade,et al.  Visual areas and spatial summation in human visual cortex , 2001, Vision Research.

[32]  C. Honey,et al.  Hierarchical process memory: memory as an integral component of information processing , 2015, Trends in Cognitive Sciences.

[33]  Kenneth A. Norman,et al.  Discovering Event Structure in Continuous Narrative Perception and Memory , 2016, Neuron.

[34]  Matthew H. Davis,et al.  Hierarchical Organization of Auditory and Motor Representations in Speech Perception: Evidence from Searchlight Similarity Analysis , 2015, Cerebral cortex.

[35]  Jonathan H. Venezia,et al.  Hierarchical organization of human auditory cortex: evidence from acoustic invariance in the response to intelligible speech. , 2010, Cerebral cortex.

[36]  Talma Hendler,et al.  Vase or face? A neural correlate of shape-selective grouping processes in the human brain , 2001, NeuroImage.

[37]  Uri Hasson,et al.  Not Lost in Translation: Neural Responses Shared Across Languages , 2012, The Journal of Neuroscience.

[38]  S. Dehaene,et al.  Cortical representation of the constituent structure of sentences , 2011, Proceedings of the National Academy of Sciences.

[39]  Jeffrey M. Zacks,et al.  Human Brain Activity Time-Locked to Narrative Event Boundaries , 2007, Psychological science.

[40]  Raksha Anand,et al.  Gist Distinctions in Healthy Cognitive Aging Versus Mild Alzheimer's Disease , 2006, Brain Impairment.

[41]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[42]  C. Honey,et al.  A place for time: the spatiotemporal structure of neural dynamics during natural audition. , 2013, Journal of neurophysiology.

[43]  David Poeppel,et al.  The analysis of speech in different temporal integration windows: cerebral lateralization as 'asymmetric sampling in time' , 2003, Speech Commun..