Differential brain mechanisms during reading human vs. machine translated fiction and news texts

Few neuroimaigng studies on reading comprehension have been conducted under natural reading settings. In this study, we showed texts presented in a natural way during functional MRI (fMRI) measurements to reveal brain areas sensitive to reading comprehension. Specifically, this paradigm independently manipulated two holistic features of article style: text genre and translation style, a qualitative index of how typical word choices and arrangements are made in daily use of the language. Specifically, articles from The New York Times (news) and Reader’s Digest (fiction) translated from English to Mandarin Chinese either by human experts or machine (Google Translate) were used to investigate the correlation of brain activity across participants during article reading. We found that bi-hemispheric visual cortex, precuneus, and occipito-parietal junction show significantly correlated hemodynamics across participants regardless of translation style and article genre. Compared to machine translation, reading human expert translation elicited more reliable fMRI signals across participants at precuneus, potentially because narrative representations and contents can be coherently presented over tens of seconds. We also found significantly stronger inter-subject correlated fMRI signals at temporal poles and fusiform gyri in fiction reading than in news reading. This may be attributed to more stable empathy processing across participants in fiction reading. The degree of stability of brain responses across subjects at extra-linguistic areas was found correlated with subjective rating on the text fluency. The functional connectivity between these areas was modulated by text genre and translation style. Taken together, our imaging results suggested stable and selective neural substrates associated with comprehending holistic features of written narratives.

[1]  Johnny T. C. Hsu,et al.  Acute lesions that impair affective empathy. , 2013, Brain : a journal of neurology.

[2]  Steve Majerus,et al.  The left intraparietal sulcus and verbal short-term memory: Focus of attention or serial order? , 2006, NeuroImage.

[3]  A. Jacobs Neurocognitive poetics: methods and models for investigating the neuronal and cognitive-affective bases of literature reception , 2015, Front. Hum. Neurosci..

[4]  Maria Luisa Gorno-Tempini,et al.  Structural anatomy of empathy in neurodegenerative disease. , 2006, Brain : a journal of neurology.

[5]  U. Hasson,et al.  Speaker–listener neural coupling underlies successful communication , 2010, Proceedings of the National Academy of Sciences.

[6]  Y. Benjamini Discovering the false discovery rate , 2010 .

[7]  D. Heeger,et al.  A Hierarchy of Temporal Receptive Windows in Human Cortex , 2008, The Journal of Neuroscience.

[8]  Robin I. M. Dunbar,et al.  The social brain hypothesis and its implications for social evolution , 2009, Annals of human biology.

[9]  Rutvik H. Desai,et al.  The neural substrates of natural reading: a comparison of normal and nonword text using eyetracking and fMRI , 2014, Front. Hum. Neurosci..

[10]  Robin I. M. Dunbar,et al.  Understanding primate brain evolution , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[11]  Keith Oatley,et al.  Fiction: Simulation of Social Worlds , 2016, Trends in Cognitive Sciences.

[12]  Feng Qi Han,et al.  Rapid learning in cortical coding of visual scenes , 2007, Nature Neuroscience.

[13]  Uri Hasson,et al.  Not Lost in Translation: Neural Responses Shared Across Languages , 2012, The Journal of Neuroscience.

[14]  Thomas Holtgraves,et al.  The role of the right hemisphere in speech act comprehension , 2012, Brain and Language.

[15]  C. Honey,et al.  Topographic Mapping of a Hierarchy of Temporal Receptive Windows Using a Narrated Story , 2011, The Journal of Neuroscience.

[16]  HighWire Press The journal of neuroscience : the official journal of the Society for Neuroscience. , 1981 .

[17]  Martijn Veltkamp,et al.  How Does Fiction Reading Influence Empathy? An Experimental Investigation on the Role of Emotional Transportation , 2013, PloS one.

[18]  Allen R. Braun,et al.  Neural correlates and network connectivity underlying narrative production and comprehension: A combined fMRI and PET study , 2014, Cortex.

[19]  John M. Henderson,et al.  Neural correlates of fixation duration in natural reading: Evidence from fixation-related fMRI , 2015, NeuroImage.

[20]  Emanuele Castano,et al.  Reading Literary Fiction Improves Theory of Mind , 2013, Science.

[21]  Arne Nagels,et al.  Feeling addressed! The role of body orientation and co‐speech gesture in social communication , 2015, Human brain mapping.

[22]  Walter Schneider,et al.  The neural correlates of strategic reading comprehension: Cognitive control and discourse comprehension , 2011, NeuroImage.

[23]  Janice Chen,et al.  Dynamic reconfiguration of the default mode network during narrative comprehension , 2016, Nature Communications.

[24]  M. Kutas,et al.  Semantic integration in reading: engagement of the right hemisphere during discourse processing. , 1999, Brain : a journal of neurology.

[25]  Robin I. M. Dunbar Social Brain Hypothesis , 1998, Encyclopedia of Evolutionary Psychological Science.

[26]  Robert Oostenveld,et al.  Cerebral coherence between communicators marks the emergence of meaning , 2013, Proceedings of the National Academy of Sciences.

[27]  Roel M. Willems,et al.  Caring About Dostoyevsky: The Untapped Potential of Studying Literature , 2016, Trends in Cognitive Sciences.

[28]  C. Frith The social brain? , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[29]  Todd B. Parrish,et al.  Inferences during Story Comprehension: Cortical Recruitment Affected by Predictability of Events and Working Memory Capacity , 2008, Journal of Cognitive Neuroscience.

[30]  Roger Ratcliff,et al.  Inference during reading. , 1992 .

[31]  M. Jung-Beeman Bilateral brain processes for comprehending natural language , 2005, Trends in Cognitive Sciences.

[32]  Rebecca Elliott,et al.  Neuronal correlates of theory of mind and empathy: A functional magnetic resonance imaging study in a nonverbal task , 2006, NeuroImage.

[33]  Arthur Gretton,et al.  Low-Frequency Local Field Potentials and Spikes in Primary Visual Cortex Convey Independent Visual Information , 2008, The Journal of Neuroscience.

[34]  Keith Oatley,et al.  The Function of Fiction is the Abstraction and Simulation of Social Experience , 2008, Perspectives on psychological science : a journal of the Association for Psychological Science.

[35]  D. Heeger,et al.  Reliability of cortical activity during natural stimulation , 2010, Trends in Cognitive Sciences.

[36]  D. Hassabis,et al.  Deconstructing episodic memory with construction , 2007, Trends in Cognitive Sciences.

[37]  F Mechler,et al.  Robust Temporal Coding of Contrast by V1 Neurons for Transient But Not for Steady-State Stimuli , 1998, The Journal of Neuroscience.

[38]  R. Fisher FREQUENCY DISTRIBUTION OF THE VALUES OF THE CORRELATION COEFFIENTS IN SAMPLES FROM AN INDEFINITELY LARGE POPU;ATION , 1915 .

[39]  R. Malach,et al.  Intersubject Synchronization of Cortical Activity During Natural Vision , 2004, Science.

[40]  Koji Jimura,et al.  Temporal pole activity during understanding other persons' mental states correlates with neuroticism trait , 2010, Brain Research.

[41]  Riitta Parkkola,et al.  Is emotional contagion special? An fMRI study on neural systems for affective and cognitive empathy , 2008, NeuroImage.

[42]  Christopher J. Honey,et al.  Selective and Invariant Neural Responses to Spoken and Written Narratives , 2013, The Journal of Neuroscience.

[43]  Jiang Xu,et al.  Language in context: emergent features of word, sentence, and narrative comprehension , 2005, NeuroImage.

[44]  John M. Henderson,et al.  Toward Semantics in the Wild: Activation to Manipulable Nouns in Naturalistic Reading , 2016, The Journal of Neuroscience.