Studying and designing emotions in live interactions with the audience

In the last decade, a number of projects have investigated the use of digital technologies to increase the engagement of the audience in live interactive performance. In this paper, we explore the relation between interactive performance and emotional participation of the audience through an experiment in the wild where perftormance is mediated by a computational system, DoPPioGioco. DoPPioGioco detects the emotional response of the audience, conveyed through face expression, allowing the performer to encourage or oppose it as the performance progresses. Based on the data collected during the experiment, we discuss the limitations and strengths of this approach for audience engagement, in the attempt to identify the factors which must be accounted for when affective technologies are employed to mediate between audience and performer in live interaction.

[1]  Ruth Aylett,et al.  I contain multitudes: creativity and emergent narrative , 2013, Creativity & Cognition.

[2]  Vincenzo Lombardo,et al.  Automatic Annotation of Characters' Emotions in Stories , 2015, ICIDS.

[3]  Antonio Chella,et al.  Conveying Audience Emotions Through Humanoid Robot Gestures to an Orchestra During a Live Musical Exhibition , 2017, CISIS.

[4]  Vincenzo Lombardo,et al.  DoppioGioco. Playing with the Audience in an Interactive Storytelling Platform , 2017, CISIS.

[5]  Robert Michael Young,et al.  Narrative Planning: Balancing Plot and Character , 2010, J. Artif. Intell. Res..

[6]  R. Plutchik.  Emotion, a psychoevolutionary synthesis , 1980 .

[7]  Celine Latulipe,et al.  Love, hate, arousal and engagement: exploring audience responses to performing arts , 2011, CHI.

[8]  Vincenzo Lombardo,et al.  Coupling conceptual modeling and rules for the annotation of dramatic media , 2015, Semantic Web.

[9]  Ruth Aylett,et al.  Emergent Narrative, Past, Present and Future of an Interactive Storytelling Approach , 2015 .

[10]  K. Scherer,et al.  Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) corpus , 2010 .

[11]  Ivo Swartjes,et al.  An Experiment in Improvised Interactive Drama , 2009, INTETAIN.

[12]  Hongsong Li,et al.  Enhancing Audience Engagement in Performing Arts Through an Adaptive Virtual Environment with a Brain-Computer Interface , 2016, IUI.

[13]  Pablo César,et al.  Co-present and remote audience experiences: intensity and cohesion , 2017, Multimedia Tools and Applications.

[14]  Chen Wang,et al.  Sensing a live audience , 2014, CHI.

[15]  Matias Valdenegro-Toro,et al.  Real-time Convolutional Neural Networks for emotion and gender classification , 2017, ESANN.

[16]  Bonnie Marranca,et al.  Performance as Design: The Mediaturgy of John Jesurun's Firefall , 2010, PAJ: A Journal of Performance and Art.

[17]  Stephanie E. Pitts,et al.  Understanding Audience Engagement in the Contemporary Arts , 2020 .

[18]  Ira J. Roseman.  Appraisal in the Emotion System: Coherence in Strategies for Coping , 2013 .

[19]  P. White.  Appraisal Theory , 2015 .

[20]  Nikitas M. Sgouros,et al.  Analysis, management and indexing of distributed multimedia performances based on audience feedback , 2003, Multimedia Systems.

[21]  Boyang Li,et al.  Story Generation with Crowdsourced Plot Graphs , 2013, AAAI.

[22]  J. G. Tanenbaum.  How I Learned to Stop Worrying and Love the Gamer: Reframing Subversive Play in Story-Based Games , 2013, DiGRA Conference.

[23]  Joe Marshall,et al.  Uncomfortable interactions , 2012, CHI.

[24]  Michael Mateas,et al.  Comparing Player Responses to Choice-Based Interactive Narratives Using Facial Expression Analysis , 2018, ICIDS.

[25]  Christopher Lindinger,et al.  The (St)Age of Participation: audience involvement in interactive performances , 2013, Digit. Creativity.

[26]  Julia Listengarten,et al.  Playing with Theory in Theatre Practice , 2011 .

[27]  Thecla Schiphorst,et al.  Audience empathy: a phenomenological method for mediated performance , 2011, C&C '11.

[28]  Ana Paiva,et al.  FearNot! - An Experiment in Emergent Narrative , 2005, IVA.

[29]  Vincenzo Lombardo,et al.  Algorithms and Interoperability between Drama and Artificial Intelligence , 2019, TDR/The Drama Review.

[30]  Vincenzo Lombardo,et al.  Artificial Intelligence for Dramatic Performance , 2019, AI*IA.

[31]  Elinor Fuchs,et al.  Presence and the Revenge of Writing: Re-thinking Theatre After Derrida , 2018 .

[32]  Vincenzo Lombardo,et al.  Thinning the Fourth Wall with Intelligent Prompt , 2017, ICIDS.

[33]  Leonardo Lesmo,et al.  Emotional range in value-sensitive deliberation , 2013, AAMAS.

[34]  Andrew Stern,et al.  Façade: An Experiment in Building a Fully-Realized Interactive Drama , 2003 .

[35]  J. Russell.  Core affect and the psychological construction of emotion. , 2003, Psychological review.

[36]  Chiara Rossitto,et al.  Understanding audience participation in an interactive theater performance , 2014, NordiCHI.

[37]  Catherine J. Stevens,et al.  Methods for measuring audience reactions , 2007 .

[38]  Brian Magerko,et al.  Narrative Development in Improvisational Theatre , 2009, ICIDS.

[39]  Sharon Lynn Chu Yew Yee,et al.  The Effects of Interactive Emotional Priming on Storytelling: An Exploratory Study , 2019, ICIDS.

[40]  Vincenzo Lombardo,et al.  All about face. An experiment in face emotion recognition in interactive dramatic performance , 2019, 2019 8th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW).

[41]  W. Tschacher,et al.  Physiological Correlates of Aesthetic Perception of Artworks in a Museum , 2012 .

[42]  Hongsong Li,et al.  Am I in the theater?: usability study of live performance based virtual reality , 2018, VRST.

[43]  Ruth Aylett,et al.  Distributed Drama Management: Beyond Double Appraisal in Emergent Narrative , 2012, ICIDS.

[44]  P. Silvia.  Looking past pleasure: Anger, confusion, disgust, pride, surprise, and other unusual aesthetic emotions. , 2009 .