CES-533: Analysis of the Event-related Potentials induced by cuts in feature movies and evaluation of the possibility of using such ERPs for understanding the effects of cuts on viewers

In this paper, we analyse the Event-Related Potentials (ERPs) produced by cuts where the scenes before and after the cut are narratively related. In tests with 6 participants and 930 cuts from 5 Hollywood feature movies we found that cuts produce a large negative ERP with an onset 100 ms after a cut and a duration of 600 ms, distributed over a very large region of the scalp. The real-world nature of the stimuli makes it hard to characterise the effects of cuts on a trial-by-trial basis. However, we found that aggregating data across all electrodes and averaging the ERPs elicited by cuts across all participants (a technique we borrowed from collaborative brain-computer interfaces) produced more reliable information. In particular we were able to reveal a relationship between the length of shots and the amplitude of the corresponding ERP with longer scenes producing bigger amplitudes. We also found that amplitudes vary across and within movies, most likely as a consequence of movie directors and editors using different choices of cutting techniques. In the future, we will explore the possibility of turning these ?findings into a collaborative brain-computer interface for aiding test screening by evaluating whether specific cuts have their intended effect on viewers.

[1]  Tzyy-Ping Jung,et al.  A Collaborative Brain-Computer Interface for Improving Human Performance , 2011, PloS one.

[2]  Byron Reeves,et al.  WATCHING TELEVISION , 1986 .

[3]  Riccardo Poli,et al.  Collaborative brain-computer interfaces for the automatic classification of images , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[4]  R. Goldstein,et al.  Attention to television: intrastimulus effects of movement and scene changes on alpha variation over time. , 1985, The International journal of neuroscience.

[5]  R. Goldstein,et al.  EEG ACTIVITY AND THE PROCESSING OF TELEVISION COMMERCIALS , 1986 .

[6]  Hiroshi Nittono,et al.  Electrophysiology of Kansei: Recent Advances in Event-Related Brain Potential Research , 2008 .

[7]  T. Smith The attentional theory of cinematic continuity , 2012 .

[8]  Riccardo Poli,et al.  Collaborative Brain-Computer Interface for Aiding Decision-Making , 2014, PloS one.

[9]  Vincent M. Reid,et al.  N400 involvement in the processing of action sequences , 2008, Neuroscience Letters.

[10]  Thomas Martinetz,et al.  Variability of eye movements when viewing dynamic natural scenes. , 2010, Journal of vision.

[11]  Phillip J. Holcomb,et al.  Two Neurocognitive Mechanisms of Semantic Integration during the Comprehension of Visual Real-world Events , 2008, Journal of Cognitive Neuroscience.

[12]  Jeffrey Nichols,et al.  Proceedings of the 2013 international conference on Intelligent user interfaces , 1998, IUI 2013.

[13]  Piotr Francuz,et al.  Does the Brain Differentiate Between Related and Unrelated Cuts When Processing Audiovisual Messages? An ERP Study , 2013 .

[14]  Peng Yuan,et al.  Study on an online collaborative BCI to accelerate response to visual targets , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[15]  Alan Gevins,et al.  Attention and Brain Activity While Watching Television: Components of Viewer Engagement , 2004 .

[16]  Annie Lang,et al.  The effects of production pacing and arousing content on the information processing of television messages , 1999 .

[17]  Annie Lang,et al.  The Effects of Edits on Arousal, Attention, and Memory for Television Messages: When an Edit Is an Edit Can an Edit Be Too Much? , 2000 .

[18]  Annie Lang,et al.  The Effects of Related and Unrelated Cuts on Television Viewers' Attention, Processing Capacity, and Memory , 1993 .

[19]  Kara D. Federmeier,et al.  Thirty years and counting: finding meaning in the N400 component of the event-related brain potential (ERP). , 2011, Annual review of psychology.

[20]  Riccardo Poli,et al.  Towards cooperative brain-computer interfaces for space navigation , 2013, IUI '13.

[21]  Riccardo Poli,et al.  Collaborative brain-computer interfaces for target localisation in rapid serial visual presentation , 2014, 2014 6th Computer Science and Electronic Engineering Conference (CEEC).

[22]  J. Henderson,et al.  Edit Blindness: The relationship between attention and global change blindness in dynamic scenes. , 2008 .