Computational Lens on Cognition: Study Of Autobiographical Versus Imagined Stories With Large-Scale Language Models

Lifelong experiences and learned knowledge lead to shared expectations about how common situations tend to unfold. Such knowledge enables people to interpret story narratives and identify salient events effortlessly. We study differences in the narrative flow of events in autobiographical versus imagined stories using GPT-3, one of the largest neural language models created to date. The diary-like stories were written by crowdworkers about either a recently experienced event or an imagined event on the same topic. To analyze the narrative flow of events of these stories, we measured sentence sequentiality, which compares the probability of a sentence with and without its preceding story context. We found that imagined stories have higher sequentiality than autobiographical stories, and that the sequentiality of autobiographical stories is higher when they are retold than when freshly recalled. Through an annotation of events in story sentences, we found that the story types contain similar proportions of major salient events, but that the autobiographical stories are denser in factual minor events. Furthermore, in comparison to imagined stories, autobiographical stories contain more concrete words and words related to the first person, cognitive processes, time, space, numbers, social words, and core drives and needs. Our findings highlight the opportunity to investigate memory and cognition with large-scale statistical language models.

[1]  Philippe Laban,et al.  Can Transformer Models Measure Coherence In Text: Re-Thinking the Shuffle Test , 2021, ACL.

[2]  Tal August,et al.  All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text , 2021, ACL.

[3]  Olivier Toubia,et al.  How quantifying the shape of stories predicts their success , 2021, Proceedings of the National Academy of Sciences.

[4]  Samuel A. Nastase,et al.  Thinking ahead: spontaneous prediction in context as a keystone of language in humans and machines , 2020, bioRxiv.

[5]  Kate G. Blackburn,et al.  The narrative arc: Revealing core narrative structures through text analysis , 2020, Science Advances.

[6]  Eric Horvitz,et al.  Recollection versus Imagination: Exploring Human Memory and Cognition via Neural Language Models , 2020, ACL.

[7]  Mark Chen,et al.  Language Models are Few-Shot Learners , 2020, NeurIPS.

[8]  Péter Simor,et al.  Novelty Manipulations, Memory Performance, and Predictive Coding: the Role of Unexpectedness , 2020, Frontiers in Human Neuroscience.

[9]  T. Underwood Machine Learning and Human Perspective , 2020, PMLA/Publications of the Modern Language Association of America.

[10]  Jeffrey M. Zacks,et al.  Event Perception and Memory. , 2019, Annual review of psychology.

[11]  David Bamman,et al.  Literary Event Detection , 2019, ACL.

[12]  Robert T. Knight,et al.  Medial Orbitofrontal Cortex, Dorsolateral Prefrontal Cortex, and Hippocampus Differentially Represent the Event Saliency , 2019, Journal of Cognitive Neuroscience.

[13]  Robert T. Knight,et al.  Event segmentation reveals working memory forgetting rate , 2019, bioRxiv.

[14]  L. Davachi,et al.  Transcending time in the brain: How event memories are constructed from experience , 2019, Hippocampus.

[15]  Samuel J. Gershman,et al.  Structured event memory: a neuro-symbolic model of event cognition , 2019, bioRxiv.

[16]  Ilya Sutskever,et al.  Language Models are Unsupervised Multitask Learners , 2019 .

[17]  A. Mendelsohn,et al.  Autobiographical memory: From experiences to brain representations , 2017, Neuropsychologia.

[18]  Alec Radford,et al.  Improving Language Understanding by Generative Pre-Training , 2018 .

[19]  David E. Rumelhart,et al.  The Representation of Knowledge in Memory 1 , 2017 .

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  Christopher M. Danforth,et al.  The emotional arcs of stories are dominated by six basic shapes , 2016, EPJ Data Science.

[22]  C. Fioretti,et al.  Why Narrating Changes Memory: A Contribution to an Integrative Model of Memory and Narrative Processes , 2016, Integrative psychological & behavioral science.

[23]  Amy Beth Warriner,et al.  Concreteness ratings for 40 thousand generally known English word lemmas , 2014, Behavior research methods.

[24]  Benjamin Van Durme,et al.  Reporting bias and knowledge acquisition , 2013, AKBC '13.

[25]  Boyang Li,et al.  Story Generation with Crowdsourced Plot Graphs , 2013, AAAI.

[26]  R. Henson,et al.  How schema and novelty augment memory formation , 2012, Trends in Neurosciences.

[27]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[28]  Charles Kemp,et al.  Bayesian models of cognition , 2008 .

[29]  R. Levy Expectation-based syntactic comprehension , 2008, Cognition.

[30]  Jeffrey M. Zacks,et al.  Segmentation in the perception and memory of events , 2008, Trends in Cognitive Sciences.

[31]  Jeffrey M. Zacks,et al.  Event perception: a mind-brain perspective. , 2007, Psychological bulletin.

[32]  Sharron E. Whitecross,et al.  Neurophysiological correlates of memory for experienced and imagined events , 2003, Neuropsychologia.

[33]  John Hale,et al.  A Probabilistic Earley Parser as a Psycholinguistic Model , 2001, NAACL.

[34]  E. Loftus,et al.  Errors in autobiographical memory. , 1998, Clinical psychology review.

[35]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[36]  A. Stone,et al.  Emotional expression and physical health: revising traumatic memories or fostering self-regulation? , 1996, Journal of personality and social psychology.

[37]  Stephen J. Anderson,et al.  Recollections of true and false autobiographical memories. , 1996 .

[38]  Kenneth Ward Church,et al.  Word Association Norms, Mutual Information, and Lexicography , 1989, ACL.

[39]  W. Kintsch The role of knowledge in discourse comprehension: a construction-integration model. , 1988, Psychological review.

[40]  John B. Black,et al.  Causal coherence and memory for events in narratives , 1981 .

[41]  L R Squire,et al.  Two forms of human amnesia: an analysis of forgetting , 1981, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[42]  Arthur C. Graesser,et al.  Incorporating inferences in narrative representations: A study of how and why , 1981, Cognitive Psychology.

[43]  John B. Black,et al.  Scripts in memory for text , 1979, Cognitive Psychology.

[44]  “Schooling and the acquisition of knowledge” , 1979 .

[45]  W. Kintsch,et al.  The role of culture‐specific schemata in the comprehension and recall of stories∗ , 1978 .

[46]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[47]  P. Thorndyke Cognitive structures in comprehension and memory of narrative discourse , 1977, Cognitive Psychology.

[48]  E. Tulving,et al.  Episodic and semantic memory , 1972 .

[49]  F. Bartlett,et al.  Remembering: A Study in Experimental and Social Psychology , 1932 .