Changes in situation models modulate processes of event perception in audiovisual narratives.

Humans understand text and film by mentally representing their contents in situation models. These describe situations using dimensions like time, location, protagonist, and action. Changes in 1 or more dimensions (e.g., a new character enters the scene) cause discontinuities in the story line and are often perceived as boundaries between 2 meaningful units. Recent theoretical advances in event perception led to the assumption that situation models are represented in the form of event models in working memory. These event models are updated at event boundaries. Points in time at which event models are updated are important: Compared with situations during an ongoing event, situations at event boundaries are remembered more precisely and predictions about what happens next become less reliable. We hypothesized that these effects depend on the number of changes in the situation model. In 2 experiments, we had participants watch sitcom episodes and measured recognition memory and prediction performance for event boundaries that contained a change in 1, 2, 3, or 4 dimensions. Results showed a linear relationship: the more dimensions changed, the higher recognition performance was. At the same time, participants' predictions became less reliable with an increasing number of dimension changes. These results suggest that updating of event models at event boundaries occurs incrementally.

[1]  Markus Huff,et al.  Changing Viewpoints during Dynamic Events , 2007, Perception.

[2]  J. Zacks,et al.  Using movement and intentions to understand human activity , 2009, Cognition.

[3]  Jeffrey M. Zacks,et al.  Temporal changes as event boundaries: Processing and memory consequences of narrative time shifts , 2005 .

[4]  Jeffrey M. Zacks,et al.  A Computational Model of Event Segmentation From Perceptual Prediction , 2007, Cogn. Sci..

[5]  Jeffrey M. Zacks,et al.  Event perception: a mind-brain perspective. , 2007, Psychological bulletin.

[6]  Stephan Schwan,et al.  The Role of Segmentation in Perception and Understanding of Events , 2008 .

[7]  H. Schriefers,et al.  Prediction in language comprehension beyond specific words: An ERP study on sentence comprehension in Polish , 2013 .

[8]  Jeffrey M. Zacks,et al.  Event understanding and memory in healthy aging and dementia of the Alzheimer type. , 2006, Psychology and aging.

[9]  Markus Huff,et al.  Spatial updating of dynamic scenes: Tracking multiple invisible objects across viewpoint changes , 2010, Attention, perception & psychophysics.

[10]  Darren Newtson,et al.  The perceptual organization of ongoing behavior , 1976 .

[11]  R. Harald Baayen,et al.  Analyzing linguistic data: a practical introduction to statistics using R, 1st Edition , 2008 .

[12]  Patricia Baggett,et al.  Role of temporal overlap of visual and auditory material in forming dual media associations. , 1984 .

[13]  Jonathan Westley Peirce,et al.  Neuroinformatics Original Research Article Generating Stimuli for Neuroscience Using Psychopy , 2022 .

[14]  Rolf A. Zwaan,et al.  Indexing space and time in film understanding , 2001 .

[15]  G. Lassiter.  Behavior Perception, Affect, and Memory , 1988 .

[16]  Herre van Oostendorp,et al.  Do readers construct spatial representations in naturalistic story comprehension , 1993 .

[17]  Barbara Tversky,et al.  Structuring information interfaces for procedural learning. , 2003, Journal of experimental psychology. Applied.

[18]  Filip Germeys,et al.  The psychology of film: perceiving beyond the cut , 2007, Psychological research.

[19]  H. Taylor,et al.  Not all narrative shifts function equally , 2000, Memory & cognition.

[20]  G. R. Potts,et al.  Assessing the occurrence of elaborative inferences: Lexical decision versus naming , 1988 .

[21]  W Hirst,et al.  On the representation of events: a study of orientation, recall, and recognition. , 1989, Journal of experimental psychology. General.

[22]  Joseph P. Magliano,et al.  The Impact of Continuity Editing in Narrative Film on Event Segmentation , 2011, Cogn. Sci..

[23]  Joseph P. Magliano,et al.  Aging and perceived event structure as a function of modality , 2012, Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition.

[24]  Jeffrey M. Zacks,et al.  Human brain activity time-locked to perceptual event boundaries , 2001, Nature Neuroscience.

[25]  Markus Huff,et al.  IBES: a tool for creating instructions based on event segmentation , 2013, Front. Psychol..

[26]  J. Henderson,et al.  Edit Blindness: The relationship between attention and global change blindness in dynamic scenes. , 2008 .

[27]  Jeffrey M. Zacks,et al.  Prediction Error Associated with the Perceptual Segmentation of Naturalistic Events , 2011, Journal of Cognitive Neuroscience.

[28]  Jeffrey M. Zacks,et al.  Event boundaries in perception affect memory encoding and updating. , 2009, Journal of experimental psychology. General.

[29]  Jodie A. Baird,et al.  Making sense of human behavior: Action parsing and intentional inference , 2001 .

[30]  Mike Rinck,et al.  Who when where: An experimental test of the event-indexing model , 2003, Memory & cognition.

[31]  R. Barker,et al.  One boy's day : a specimen record of behavior , 1951 .

[32]  Rolf A. Zwaan,et al.  The Construction of Situation Models in Narrative Comprehension: An Event-Indexing Model , 1995 .

[33]  W. Hirst,et al.  Recognizing differences in recognition tasks: A reply to Lassiter and Slaw. , 1991 .

[34]  Jeffrey M. Zacks,et al.  Segmentation in reading and film comprehension. , 2009, Journal of experimental psychology. General.

[35]  Rolf A. Zwaan,et al.  Constructing Multidimensional Situation Models During Reading , 1998 .

[36]  Christopher A. Kurby,et al.  Starting from scratch and building brick by brick in comprehension , 2012, Memory & Cognition.

[37]  Markus Huff,et al.  Tracking multiple objects across abrupt viewpoint changes , 2009 .

[38]  Scott L. Rogers,et al.  Memorial consequences of variation in behavior perception , 1988 .

[39]  Stephan Schwan,et al.  Watching Film for the First Time , 2010, Psychological science.

[40]  Markus Huff,et al.  Visual target detection is impaired at event boundaries , 2012 .

[41]  Rolf A. Zwaan,et al.  Situation models in language comprehension and memory. , 1998, Psychological bulletin.

[42]  W. Kintsch,et al.  Strategies of discourse comprehension , 1983 .

[43]  Arthur C. Graesser,et al.  Dimensions of situation model construction in narrative comprehension. , 1995 .

[44]  Darren Newtson.  Attribution and the unit of perception of ongoing behavior. , 1973 .

[45]  Rolf A. Zwaan.  Situation Models , 1999 .

[46]  Neil Cohn,et al.  Prediction, events, and the advantage of Agents: The processing of semantic roles in visual narrative , 2013, Cognitive Psychology.

[47]  Markus Huff,et al.  Do not cross the line: Heuristic spatial updating in dynamic scenes , 2012, Psychonomic bulletin & review.

[48]  G. Daniel Lassiter,et al.  The unitization and memory of events , 1991 .

[49]  S. Schwan,et al.  The cognitive representation of filmic event summaries , 2004 .