Towards Multisensory Storytelling with Taste and Flavor

Film makers, producers, and theaters have continuously looked at ways to embody and/or integrate multiple sensory cues in the experiences they deliver. Here, we present a reflection on past attempts, lessons learnt, and future directions for the community around multisensory TV, film, and multimedia as a historical, though renewed, space of content creation. In particular, we present an overview of what we call "tasty film", that is, film involving taste, flavor, and more broadly food and drink inputs, to influence the audience experience. We suggest that such elements should be considered beyond "add-ons" in film experiences. We advocate for experimentation with new kinds of storytelling taking inspiration from multisensory design research and work on sensory substitution. We position this article as a starting point for anyone interested in multisensory film involving taste, flavor, and foods.

[1]  John Prescott,et al.  Multisensory processes in flavour perception and their influence on food choice , 2015 .

[2]  J. Costa.  A brave new world , 2018, Virchows Archiv.

[3]  Chi Thanh Vi,et al.  Multisensory Experiences in HCI , 2017, IEEE MultiMedia.

[4]  A. Hirsch,et al.  What the Nose Knows: The Science of Scent in Everyday Life , 2009 .

[5]  Weisi Lin,et al.  Mulsemedia: State of the Art, Perspectives, and Challenges , 2014, TOMM.

[6]  C. Spence.  Multisensory Flavor Perception , 2015, Cell.

[7]  Gabriel-Miro Muntean,et al.  Olfaction-Enhanced Multimedia , 2016, ACM Comput. Surv..

[8]  Carlos Velasco,et al.  Integrating Mid-Air Haptics into Movie Experiences , 2017, TVX.

[9]  Rob Comber,et al.  Temporal, affective, and embodied characteristics of taste experiences: a framework for design , 2014, CHI.

[10]  Chi Thanh Vi,et al.  TastyFloats: A Contactless Food Delivery System , 2017, ISS.

[11]  Sarah Atkinson,et al.  Inside-the-scenes: The rise of experiential cinema , 2016 .

[12]  Brian Yecies.  Transnational collaboration of the multisensory kind: Exploiting Korean 4D cinema in China , 2016 .

[13]  Sergio Casas,et al.  ROMOT: A Robotic 3D-Movie Theater Allowing Interaction and Multimodal Experiences , 2016, LSR.

[14]  Charles Spence,et al.  Colour-taste correspondences: Designing food experiences to meet expectations or to surprise , 2016 .

[15]  D. Neumark-Sztainer,et al.  Associations between watching TV during family meals and dietary intake among adolescents. , 2007, Journal of nutrition education and behavior.

[16]  J. Delwiche,et al.  Are there ‘basic’ tastes? , 1996 .

[17]  Chi Thanh Vi,et al.  Not just seeing, but also feeling art: Mid-air haptic experiences integrated in a multisensory art exhibition , 2017, Int. J. Hum. Comput. Stud..

[18]  Gabriel-Miro Muntean,et al.  Beyond Multimedia Adaptation: Quality of Experience-Aware Multi-Sensorial Media Delivery , 2015, IEEE Transactions on Multimedia.

[19]  Qiong Luo,et al.  Data Edibilization: Representing Data with Food , 2016, CHI Extended Abstracts.

[20]  Paul J. Laurienti,et al.  Semantic congruence is a critical factor in multisensory behavioral performance , 2004, Experimental Brain Research.

[21]  Daniela Stehlik,et al.  A Brave New World? , 2001 .

[22]  P. Bach-y-Rita,et al.  Sensory substitution and the human–machine interface , 2003, Trends in Cognitive Sciences.

[23]  John A. Bargh,et al.  Television Viewing and Unhealthy Diet: Implications for Children and Media Interventions , 2009, Health communication.

[24]  Hiroshi Ishida,et al.  Smelling screen: Presenting a virtual odor source on a LCD screen , 2013, 2013 IEEE Virtual Reality (VR).

[25]  Hiroshi Ishida,et al.  Smelling screen: Technique to present a virtual odor source at an arbitrary position on a screen , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[26]  Marianna Obrist,et al.  "I Always Wanted to See the Night Sky": Blind User Preferences for Sensory Substitution Devices , 2016, CHI.

[27]  Hiroshi Ishida,et al.  Smelling Screen: Development and Evaluation of an Olfactory Display System for Presenting a Virtual Odor Source , 2013, IEEE Transactions on Visualization and Computer Graphics.

[28]  Takuji Narumi,et al.  Meta cookie , 2010, SIGGRAPH Emerging Technologies.

[29]  J. E. Steiner,et al.  Comparative expression of hedonic impact: affective reactions to taste by human infants and other primates , 2001, Neuroscience & Biobehavioral Reviews.

[30]  Rudolf Arnheim.  Film as art , 1957 .

[31]  M. Beugnet.  Film Theory: an Introduction Through the Senses , 2012 .

[32]  Carlos Velasco,et al.  Digitizing the chemical senses: Possibilities & pitfalls , 2017, Int. J. Hum. Comput. Stud..

[33]  T. Elsaesser,et al.  Film Theory: An Introduction Through the Senses , 2009 .

[34]  C. Montell,et al.  Calcium Taste Avoidance in Drosophila , 2018, Neuron.

[35]  B. Trivedi,et al.  Neuroscience: Hardwired for taste , 2012, Nature.

[36]  Laura U. Marks,et al.  The Skin of the Film: Intercultural Cinema, Embodiment, and the Senses , 2000 .

[37]  L. Piatti-Farnell.  Approaching Food and Horror , 2017 .