A Scientific Look at the Design of Aesthetically and Emotionally Engaging Interactive Entertainment Experiences

The interactive entertainment industry has become a multi-billion dollar industry with revenues overcoming those of the movie industry (ESA, 2009). Beyond the demand for high fidelity graphics or stylized imagery, participants in these environments have come to expect certain aesthetic and artistic qualities that engage them at a very deep emotional level. These qualities pertain to the visual aesthetic, dramatic structure, pacing, and sensory systems embedded within the experience. All these qualities are carefully crafted by the creator of the interactive experience to evoke affect. In this book chapter, we will attempt to discuss the design techniques developed by artists to craft such emotionally engaging experiences. In addition, we take a scientific approach whereby we discuss case studies of the use of these design techniques and experiments that attempt to validate their use in stimulating emotions.

[1]  J. Russell.  A circumplex model of affect. , 1980 .

[2]  P. Vorderer,et al.  Media entertainment: The psychology of its appeal. , 2000 .

[3]  Robert D. Ward,et al.  Physiological responses to different WEB page designs , 2003, Int. J. Hum. Comput. Stud..

[4]  Jacquelyn Ford Morie,et al.  The Fidelity of 'Feel': Emotional Affordance in Virtual Environments , 2005 .

[5]  David Sander,et al.  A systems approach to appraisal mechanisms in emotion , 2005, Neural Networks.

[6]  Lennart E. Nacke,et al.  Log Who's Playing: Psychophysiological Game Analysis Made Easy through Event Logging , 2008, Fun and Games.

[7]  N. Ravaja,et al.  Psychophysiology of digital game playing: The relationship of self-reported emotions with phasic physiological responses , 2008 .

[8]  Robert J. K. Jacob,et al.  Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy , 2009, CHI.

[9]  B. Parkinson,et al.  Emotion and motivation , 1995 .

[10]  Dennis R. Wixon,et al.  Using the RITE method to improve products; a definition and a case study , 2007 .

[11]  R. Bornstein,et al.  Perception without Awareness: Cognitive, Clinical and Social Perspectives , 1993 .

[12]  The Past Recaptured , 1932 .

[13]  F. Thomas,et al.  The illusion of life : Disney animation , 1981 .

[14]  P. Lang.  The emotion probe. Studies of motivation and attention. , 1995, The American psychologist.

[15]  Jeremy Birn,et al.  Digital Lighting & Rendering , 2000 .

[16]  Vladimir Propp,et al.  Morphology of the folktale , 1959 .

[17]  Bruce Crowther,et al.  Film Noir: Reflections in a Dark Mirror , 1988 .

[18]  David Bordwell,et al.  Film Art: An Introduction , 1979 .

[19]  John L. Sibert,et al.  Heart rate variability: indicator of user state as an aid to human-computer interaction , 1998, CHI.

[20]  Jonas Heide Smith,et al.  Player talk—the functions of communication in multplayer role-playing games , 2008, CIE.

[21]  Richard Bryant,et al.  Tonic and Phasic Heart Rate as Predictors of Posttraumatic Stress Disorder , 2007, Psychosomatic medicine.

[22]  Andrew T. Duchowski,et al.  Eye Tracking Methodology: Theory and Practice , 2003, Springer London.

[23]  Regan L. Mandryk,et al.  Physiological indicators for the evaluation of co-located collaborative play , 2004, CSCW.

[24]  John Fitchen,et al.  The Construction of Gothic Cathedrals: A Study of Medieval Vault Erection , 1981 .

[25]  Jon Boorstin.  Making Movies Work: Thinking Like a Filmmaker , 1995 .

[26]  Michael Meehan,et al.  Physiological measures of presence in stressful virtual environments , 2002, SIGGRAPH.

[27]  N. Ravaja.  Contributions of Psychophysiology to Media Research: Review and Recommendations , 2004 .

[28]  Dave Beck,et al.  Theatre of the Oppressed , 2010 .

[29]  Joseph Campbell,et al.  The Hero with a Thousand Faces , 1949 .

[30]  Mathieu Lagrange,et al.  Enhanced Partial Tracking Using Linear Prediction , 2003 .

[31]  Charlotte Sennersten.  Gameplay (3D Game Engine + Ray Tracing = Visual Attention through Eye Tracking) , 2008 .

[32]  R. Cabeza,et al.  Imaging Cognition II: An Empirical Review of 275 PET and fMRI Studies , 2000, Journal of Cognitive Neuroscience.

[33]  D. Kort,et al.  D3.3 : Game Experience Questionnaire:development of a self-report measure to assess the psychological impact of digital games , 2007 .

[34]  E. Gibson,et al.  The "visual cliff". , 1960, Scientific American.

[35]  Robert Scholes,et al.  Elements of Drama , 1986 .

[36]  J. Lagopoulos.  Electrodermal activity , 2007, Acta Neuropsychiatrica.

[37]  K. Mathiak,et al.  Toward brain correlates of natural behavior: fMRI during violent video games , 2006, Human brain mapping.

[38]  Marc Hassenzahl,et al.  The Semantics of Fun: Differentiating Enjoyable Eeperiences , 2005, Funology.

[39]  L. Aftanas,et al.  Human anterior and frontal midline theta and lower alpha reflect emotionally positive state and internalized attention: high-resolution EEG investigation of meditation , 2001, Neuroscience Letters.

[40]  K. Iyer,et al.  Sensory design for virtual environments , 2003, SIGGRAPH '03.

[41]  Nilanjan Sarkar,et al.  Online stress detection using psychophysiological signals for implicit human-robot cooperation , 2002, Robotica.

[42]  Pieter Desmet,et al.  Measuring Emotion: Development and Application of an Instrument to Measure Emotional Responses to Products , 2005, Funology.

[43]  Uwe Flick,et al.  Designing Qualitative Research , 2008 .

[44]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[45]  Robert L. Benedetti.  The Actor at Work , 1970 .

[46]  Neerincx,et al.  Inducing and measuring emotion through a multiplayer first-person shooter computer game , 2007 .

[47]  Peter C. Wright,et al.  Funology: from usability to enjoyment , 2005 .

[48]  Bruce Phillips,et al.  Tracking real-time user experience (TRUE): a comprehensive instrumentation solution for complex systems , 2008, CHI.

[49]  Christine L. Lisetti,et al.  Proceedings of the 13th International Conference on Human Computer Interaction , 2009 .

[50]  Jennie Hall,et al.  DRAMA , 1912, Francis W. Parker School Yearbook.

[51]  Katherine Isbister,et al.  Game Usability: Advancing the Player Experience , 2008 .

[52]  Penelope M. Sanderson,et al.  Exploratory sequential data analysis: exploring continuous observational data , 1996, INTR.

[53]  C. Rennie,et al.  Decomposing skin conductance into tonic and phasic components. , 1997, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[54]  Blain Brown,et al.  Motion Picture and Video Lighting , 2007 .

[55]  Regan L. Mandryk,et al.  A continuous and objective evaluation of emotional experience with interactive play environments , 2006, CHI.

[56]  K. Poels,et al.  "It is always a lot of fun!": exploring dimensions of digital game experience using focus group methodology , 2007, Future Play.

[57]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[58]  Lennart E. Nacke,et al.  Affective Ludology: Scientific Measurement of User Experience in Interactive Entertainment , 2009 .

[59]  G Pfurtscheller,et al.  Event-related beta synchronization after wrist, finger and thumb movement. , 1998, Electroencephalography and clinical neurophysiology.

[60]  Christoph Braun,et al.  Coherence of gamma-band EEG activity as a basis for associative learning , 1999, Nature.

[61]  P. Vorderer,et al.  Video Games and the Pleasures of Control , 2000 .

[62]  Kenton O'Hara,et al.  Social Impact , 2019, Encyclopedia of Food and Agricultural Ethics.

[63]  W. Ray,et al.  EEG alpha activity reflects attentional demands, and beta activity reflects emotional and cognitive processes. , 1985, Science.

[64]  Georgios N. Yannakakis,et al.  Player modeling using self-organization in Tomb Raider: Underworld , 2009, 2009 IEEE Symposium on Computational Intelligence and Games.

[65]  Mike Kuniavsky,et al.  Observing the User Experience , 2003 .

[66]  K. Kallinen,et al.  Phasic Emotional Reactions to Video Game Events: A Psychophysiological Investigation , 2006 .

[67]  P. Ekman.  An argument for basic emotions , 1992 .

[68]  Robert S. Kennedy,et al.  Simulator Sickness Questionnaire: An enhanced method for quantifying simulator sickness. , 1993 .

[69]  Randy J. Pagulayan,et al.  User-centered design in games , 2012 .

[70]  Bill Kapralos,et al.  Proceedings of the 2007 conference on Future Play, Toronto, ON, Canada, November 15 - 17, 2007 , 2007, Future Play.

[71]  Kevin G. Suffern,et al.  Painting with light , 2002, SIGGRAPH '02.

[72]  Donald A. Norman,et al.  Emotional design , 2004, UBIQ.

[73]  Monson H. Hayes,et al.  Statistical Digital Signal Processing and Modeling , 1996 .

[74]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[75]  J. Michael Gillette,et al.  Designing with Light , 2002 .

[76]  Marietta S. Millet,et al.  Light Revealing Architecture , 1996 .

[77]  Veikko Surakka,et al.  Emotions and heart rate while sitting on a chair , 2005, CHI.

[78]  A. J. Fridlund,et al.  Guidelines for human electromyographic research. , 1986, Psychophysiology.