A Model of Event Knowledge

Our knowledge of events and situations in the world plays a critical role in our ability to understand what is happening around us, to predict what might happen next, and to comprehend language. What has not been so clear is the form and structure of this knowledge, how it is learned, and how it is deployed in real time. Despite many important theoretical proposals, often using different terminology such as schemas, scripts, frames, and event knowledge, developing a model that addresses these three questions (the form, learning, and use of such knowledge) has remained an elusive challenge for decades. In this article, we present a connectionist model of event knowledge that attempts to fill this gap. From sequences of activities, the model learns both the internal structure of activities as well as the temporal structure that organizes activity sequences. The model simulates a wide range of human behaviors that have been argued to involve the use of event knowledge and the temporal structure of events. Furthermore, it makes testable predictions about behaviors not yet observed. Most importantly, the model’s ability to learn event structure from experience is a novel solution to the question, “What is the form and representation of event knowledge?”

[1]  Brian J. Scholl,et al.  Chasing vs. Stalking: Interrupting the Perception of Animacy , 2010 .

[2]  W. Kintsch,et al.  Strategies of discourse comprehension , 1986 .

[3]  J. D. Murray,et al.  Forward Inferences in Narrative Text , 1993 .

[4]  Jeffrey M. Zacks,et al.  A Computational Model of Event Segmentation From Perceptual Prediction , 2007, Cogn. Sci..

[5]  J. Elman,et al.  Learning to use words: Event-related potentials index single-shot contextual word learning , 2010, Cognition.

[6]  J. H. Neely Semantic priming effects in visual word recognition: A selective review of current findings and theories. , 1991 .

[7]  J. Elman,et al.  Effects of event knowledge in processing verbal arguments. , 2010, Journal of memory and language.

[8]  M. D. Rutherford,et al.  Social perception : detection and interpretation of animacy, agency, and intention , 2013 .

[9]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[10]  Marta Kutas,et al.  When temporal terms belie conceptual order , 1998, Nature.

[11]  Mark A. McDaniel,et al.  The time course and durability of predictive inferences , 1993 .

[12]  G. R. Potts,et al.  Assessing the occurrence of elaborative inferences: Lexical decision versus naming , 1988 .

[13]  Marvin Minsky,et al.  A framework for representing knowledge , 1974 .

[14]  G. Dell,et al.  Representation of serial order in speech: evidence from the repeated phoneme effect in speech errors. , 1984, Journal of experimental psychology. Learning, memory, and cognition.

[15]  Albert T. Corbett,et al.  Instrument inferences and verb schemata , 1982 .

[16]  L. Barsalou,et al.  Multiple organisations of events in memory. , 1997, Memory.

[17]  Ken McRae,et al.  People Use their Knowledge of Common Events to Understand Language, and Do So as Quickly as Possible , 2009, Lang. Linguistics Compass.

[18]  P. Johnson-Laird,et al.  Mental Models: Towards a Cognitive Science of Language, Inference, and Consciousness , 1985 .

[19]  D. Rumelhart NOTES ON A SCHEMA FOR STORIES , 1975 .

[20]  Michael C. Frank,et al.  Perceptual Development in Infancy as the Foundation of Event Perception , 2008 .

[21]  Matthew W. Crocker,et al.  Connectionist Semantic Systematicity in Language Production , 2016, CogSci.

[22]  J. Bransford,et al.  Sentence memory: A constructive versus interpretive approach ☆ ☆☆ , 1972 .

[23]  Paul van den Broek,et al.  Causal Inferences and The Comprehension of Narrative Texts , 1990 .

[24]  Lawrence A. Shapiro What is Psychophysics? , 1994, PSA: Proceedings of the Biennial Meeting of the Philosophy of Science Association.

[25]  Mark Steedman,et al.  Temporal Ontology and Temporal Reference , 1988, CL.

[26]  K. McRae,et al.  Integrating Verbs, Situation Schemas, and Thematic Role Concepts , 2001 .

[27]  Sarah A. Gerson,et al.  The emergence of intention attribution in infancy. , 2009, The psychology of learning and motivation.

[28]  Mark F. St. John,et al.  The Story Gestalt: A Model of Knowledge-Intensive Processes in Text Comprehension , 1992, Cogn. Sci..

[29]  K Rayner,et al.  Elaborative inferencing as an active or passive process. , 1990, Journal of experimental psychology. Learning, memory, and cognition.

[30]  Ronald J. Williams,et al.  A Learning Algorithm for Continually Running Fully Recurrent Neural Networks , 1989, Neural Computation.

[31]  R. Ratcliff,et al.  Inferences about predictable events. , 1986, Journal of experimental psychology. Learning, memory, and cognition.

[32]  Leonard M. Freeman,et al.  A set of measures of centrality based upon betweenness , 1977 .

[33]  Robert Frank,et al.  Aspectual Coercion and the Online Computation of Sentential Aspect , 2000 .

[34]  Brian J. Scholl,et al.  The psychophysics of chasing: A case study in the perception of animacy , 2009, Cognitive Psychology.

[35]  Marshall R. Mayberry,et al.  Learning to Attend: A Connectionist Model of Situated Language Comprehension , 2009, Cogn. Sci..

[36]  Martin Paczynski,et al.  When Events Change Their Nature: The Neurocognitive Mechanisms Underlying Aspectual Coercion , 2014, Journal of Cognitive Neuroscience.

[37]  H. Laborit,et al.  [Experimental study]. , 1958, Bulletin mensuel - Societe de medecine militaire francaise.

[38]  James L. McClelland,et al.  Understanding normal and impaired word reading: computational principles in quasi-regular domains. , 1996, Psychological review.

[39]  James L. McClelland Capturing Gradience, Continuous Change, and Quasi‐Regularity in Sound, Word, Phrase, and Meaning , 2015 .

[40]  Marta Kutas,et al.  Verb aspect and the activation of event knowledge. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[41]  M. Gernsbacher,et al.  Accessing Sentence Participants: The Advantage of First Mention. , 1988, Journal of memory and language.

[42]  Geoffrey E. Hinton,et al.  Schemata and Sequential Thought Processes in PDP Models , 1986 .

[43]  Sabine Guéraud,et al.  Accessibility of outdated information. , 2010, Journal of experimental psychology. Learning, memory, and cognition.

[44]  J. Elman,et al.  Generalized event knowledge activation during online sentence comprehension. , 2012, Journal of memory and language.

[45]  John D. Bransford,et al.  The abstraction of linguistic ideas , 1971 .

[46]  D. Plaut,et al.  Doing without schema hierarchies: a recurrent connectionist approach to normal and impaired routine sequential action. , 2004, Psychological review.

[47]  J. Elman,et al.  A basis for generating expectancies for verbs from nouns , 2005, Memory & cognition.

[48]  Philip E. Agre,et al.  The dynamic structure of everyday life , 1988 .

[49]  Jeffrey M. Zacks,et al.  Event structure in perception and conception. , 2001, Psychological bulletin.

[50]  Lance J. Rips,et al.  Memory for routines , 1982 .

[51]  Z. Nadasdy,et al.  Taking the intentional stance at 12 months of age , 1995, Cognition.

[52]  Mary Hare,et al.  Activating event knowledge , 2009, Cognition.

[53]  A. D. Groot The range of automatic spreading activation in word priming , 1983 .

[54]  Victoria Southgate,et al.  Inferring the outcome of an ongoing novel action at 13 months. , 2009, Developmental psychology.

[55]  James L. McClelland,et al.  Semantic Cognition: A Parallel Distributed Processing Approach , 2004 .

[56]  N. S. Johnson,et al.  Remembrance of things parsed: Story structure and recall , 1977, Cognitive Psychology.

[57]  James A. Galambos Normative studies of six characteristics of our knowledge of common activities , 1983 .

[58]  Michael I. Jordan Serial Order: A Parallel Distributed Processing Approach , 1997 .

[59]  M. Tanenhaus,et al.  Modeling the Influence of Thematic Fit (and Other Constraints) in On-line Sentence Comprehension , 1998 .

[60]  Mathieu Koppen,et al.  Modeling knowledge-based inferences in story comprehension , 2003, Cogn. Sci..

[61]  T. Trabasso,et al.  Constructing inferences during narrative text comprehension. , 1994, Psychological review.

[62]  Curt Burgess,et al.  Mediated priming in the cerebral hemispheres , 2003, Brain and Cognition.

[63]  Roger C. Schank,et al.  Language and Memory , 1986, Cogn. Sci..

[64]  Elke Van der Meer,et al.  Insights Into Knowledge Representation: The Influence of Amodal and Perceptual Variables on Event Knowledge Retrieval From Memory , 2009, Cogn. Sci..

[65]  J. Grafman,et al.  An examination of daily activities and their scripts across the adult lifespan , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[66]  Geoffrey E. Hinton Learning and Applying Contextual Constraints in Sentence Comprehension , 1991 .

[67]  Donald A. Norman,et al.  The LNR approach to human information processing , 1981, Cognition.

[68]  K. Lashley The problem of serial order in behavior , 1951 .

[69]  Sung-il Kim,et al.  The Advantage of First Mention in Korean The Temporal Contributions of Syntactic, Semantic, and Pragmatic Factors , 2004, Journal of psycholinguistic research.

[70]  P. Thorndyke Cognitive structures in comprehension and memory of narrative discourse , 1977, Cognitive Psychology.

[71]  John J. Ohala,et al.  The Temporal Regulation of Speech , 1975 .

[72]  H. Kolk,et al.  Three-step priming in lexical decision , 2002, Memory & cognition.

[73]  John B. Black,et al.  Scripts in memory for text , 1979, Cognitive Psychology.

[74]  Jeffrey M. Zacks,et al.  Understanding events : from perception to action , 2008 .

[75]  F. Bartlett,et al.  Remembering: A Study in Experimental and Social Psychology , 1932 .

[76]  T. Bever,et al.  Empty categories access their antecedents during comprehension , 1988 .

[77]  Jeffrey L. Elman,et al.  Cues, Constraints, and Competition in Sentence Processing , 2004 .

[78]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[79]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[80]  Matthew M Botvinick,et al.  Short-term memory for serial order: a recurrent neural network model. , 2006, Psychological review.

[81]  D. J. Bennett,et al.  Mediated priming in younger and older adults. , 1999, Experimental aging research.

[82]  Marcia K. Johnson,et al.  Contextual prerequisites for understanding: Some investigations of comprehension and recall , 1972 .

[83]  D Swinney,et al.  The role of structure in coreference assignment during sentence comprehension , 1989, Journal of psycholinguistic research.

[84]  J. J. Guajardo,et al.  How infants make sense of intentional action , 2001 .

[85]  David C. Plaut,et al.  Quasiregularity and Its Discontents: The Legacy of the Past Tense Debate , 2014, Cogn. Sci..

[86]  R Ratcliff,et al.  Testing theories of language processing: an empirical investigation of the on-line lexical decision task. , 1994, Journal of experimental psychology. Learning, memory, and cognition.

[87]  J. Elman On the Meaning of Words and Dinosaur Bones: Lexical Knowledge Without a Lexicon , 2009, Cogn. Sci..

[88]  Paul Rodríguez,et al.  A Recurrent Neural Network that Learns to Count , 1999, Connect. Sci..

[89]  Kara D. Federmeier,et al.  A Rose by Any Other Name: Long-Term Memory Structure and Sentence Processing , 1999 .

[90]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[91]  John B. Black,et al.  STORY UNDERSTANDING AS PROBLEM-SOLVING * , 1980 .

[92]  Rolf A. Zwaan,et al.  Updating situation models. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[93]  J. Foley The co-ordination and regulation of movements , 1968 .

[94]  Elizabeth A. Olson,et al.  Eyewitness testimony. , 2003, Annual review of psychology.

[95]  H. Siegelmann Foundations of recurrent neural networks , 1993 .

[96]  H. Dreyfus What Computers Can't Do: The Limits of Artificial Intelligence , 1978 .

[97]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[98]  S. Garrod,et al.  Understanding written language: Explorations of comprehension beyond the sentence , 1981 .

[99]  K. McRae,et al.  False Recall in the Deese–Roediger–McDermott Paradigm: The Roles of Gist and Associative Strength , 2011, Quarterly journal of experimental psychology.

[100]  Ray Jackendoff,et al.  A Parallel Architecture perspective on language processing , 2007, Brain Research.

[101]  D. Rumelhart Schemata: The Building Blocks of Cognition , 2017 .

[102]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[103]  Jelena Mirkovic,et al.  Incrementality and Prediction in Human Sentence Processing , 2009, Cogn. Sci..

[104]  Terry Winograd,et al.  Understanding computers and cognition , 1986 .

[105]  D. Balota,et al.  Depth of Automatic Spreading Activation: Mediated Priming Effects in Pronunciation but Not in Lexical Decision , 1986 .

[106]  J. Mandler Stories, Scripts, and Scenes: Aspects of Schema Theory , 1984 .

[107]  F. Heider,et al.  An experimental study of apparent behavior , 1944 .

[108]  B. Scholl,et al.  9 Perceiving Animacy and Intentionality: Visual Processing or Higher-Level Judgment? , 2013 .