Multisensory Experiences: A Primer

We present a primer on multisensory experiences, the different components of this concept, as well as a reflection of its implications for individuals and society. We define multisensory experiences, illustrate how to understand them, elaborate on the role of technology in such experiences, and present the three laws of multisensory experiences, which can guide discussion on their implications. Further, we introduce the case of multisensory experiences in the context of eating and human-food interaction to illustrate how its components operationalize. We expect that this article provides a first point of contact for those interested in multisensory experiences, as well as multisensory experiences in the context of human-food interaction.

[1]  K. Harding.  I, Robot , 2017, Practical Neurology.

[2]  Mark J Shelhamer,et al.  Psychological and biological challenges of the Mars mission viewed through the construct of the evolution of fundamental human needs , 2018, Acta Astronautica.

[3]  Carlos Velasco,et al.  Multisensory Experiences , 2020 .

[4]  Anton Nijholt,et al.  3rd International Workshop on Multisensory Approaches to Human-Food Interaction , 2018, ICMI.

[5]  C. Spence,et al.  Factors affecting flavor perception in space: Does the spacecraft environment influence food intake by astronauts? , 2020, Comprehensive reviews in food science and food safety.

[6]  Charles Spence,et al.  Multisensory Flavour Perception: Blending, Mixing, Fusion, and Pairing within and between the Senses , 2020, Foods.

[7]  Wendy Ju,et al.  Next Steps for Human-Computer Integration , 2020, CHI.

[8]  Kosuke Sato,et al.  FoodFab: Creating Food Perception Illusions using Food 3D Printing , 2020, CHI.

[9]  Carlos Velasco,et al.  The Multisensory Analysis of Product Packaging Framework , 2018, Multisensory Packaging.

[10]  Monica Bordegoni,et al.  Prototyping strategies for multisensory product experience engineering , 2015, Journal of Intelligent Manufacturing.

[11]  Olivia Petit,et al.  Digital Sensory Marketing: Integrating New Technologies into Multisensory Online Experience , 2019, Journal of Interactive Marketing.

[12]  Ernst Kruijff,et al.  FaceHaptics: Robot Arm based Versatile Facial Haptics for Immersive Environments , 2020, CHI.

[13]  M. Hirose,et al.  MetaCookie+ , 2011, 2011 IEEE Virtual Reality Conference.

[14]  C. Spence.  Crossmodal correspondences: A tutorial review , 2011, Attention, perception & psychophysics.

[15]  Venkatesh Shankar,et al.  Transforming the Customer Experience through New Technologies , 2020, Journal of Interactive Marketing.

[16]  Kasper Hornbæk,et al.  SMELL SPACE , 2020, ACM Trans. Comput. Hum. Interact..

[17]  Paul Hekkert,et al.  Shifts in sensory dominance between various stages of user-product interactions. , 2010, Applied ergonomics.

[18]  Cesare V Parise,et al.  Crossmodal Correspondences: Standing Issues and Experimental Guidelines. , 2016, Multisensory research.

[19]  C. Spence,et al.  Synaesthesia: The multisensory dining experience , 2019, International Journal of Gastronomy and Food Science.

[20]  Chi Thanh Vi,et al.  Not just seeing, but also feeling art: Mid-air haptic experiences integrated in a multisensory art exhibition , 2017, Int. J. Hum. Comput. Stud..

[21]  Ernst Kruijff,et al.  Multisensory Virtual Reality Exposure Therapy , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[22]  H. Schifferstein.  The Roles of the Senses in Different Stages of Consumers’ Interactions With Food Products , 2016 .

[23]  Naresh K. Malhotra,et al.  Information and sensory overload. Information and sensory overload in psychology and marketing , 1984 .

[24]  R.I.A. Mercuri,et al.  Technology as Experience , 2005, IEEE Transactions on Professional Communication.

[25]  D. V. Byrne,et al.  The Role of Intrinsic and Extrinsic Sensory Factors in Sweetness Perception of Food and Beverages: A Review , 2019, Foods.

[26]  Damien Blackwell,et al.  Gastrophysics. The new science of eating [Book Review] , 2017 .

[27]  Gijs Huisman,et al.  1st international workshop on multi-sensorial approaches to human-food interaction (workshop summary) , 2016, ICMI.

[28]  Anton Nijholt,et al.  MHFI 2017: 2nd international workshop on multisensorial approaches to human-food interaction (workshop summary) , 2017, ICMI.

[29]  Aaron R. Seitz,et al.  Benefits of multisensory learning , 2008, Trends in Cognitive Sciences.

[30]  C. Spence,et al.  Factors influencing the choice of beer: A review. , 2020, Food research international.

[31]  R. Trotta,et al.  Communicating cosmology with multisensory metaphorical experiences , 2020 .

[32]  Celso A. S. Santos,et al.  A mulsemedia framework for delivering sensory effects to heterogeneous systems , 2019, Multimedia Systems.

[33]  Fumio Kishino,et al.  Augmented reality: a class of displays on the reality-virtuality continuum , 1995, Other Conferences.

[34]  G. Calvert,et al.  Multisensory integration: methodological approaches and emerging principles in the human brain , 2004, Journal of Physiology-Paris.

[35]  Marianna Obrist,et al.  OWidgets: A toolkit to enable smell-based experience design , 2019, Int. J. Hum. Comput. Stud..

[36]  Carlos Velasco,et al.  Towards Multisensory Storytelling with Taste and Flavor , 2018, MHFI@ICMI.

[37]  M. Naumer,et al.  Semantics and the multisensory brain: How meaning modulates processes of audio-visual integration , 2008, Brain Research.

[38]  J. Vroomen,et al.  Intersensory binding across space and time: A tutorial review , 2013, Attention, Perception, & Psychophysics.

[39]  Chi Thanh Vi,et al.  LeviSense: A platform for the multisensory integration in levitating food and insights into its effect on flavour perception , 2020, Int. J. Hum. Comput. Stud..

[40]  Katherine N. Lemon,et al.  Understanding Customer Experience Throughout the Customer Journey , 2016 .

[41]  Lining Yao,et al.  Space Food Experiences: Designing Passenger's Eating Experiences for Future Space Travel Scenarios , 2019, Front. Comput. Sci..

[42]  Carlos Flavián,et al.  The impact of virtual, augmented and mixed reality technologies on the customer experience , 2019, Journal of Business Research.

[43]  Joe Marshall,et al.  The Challenges of Visual-Kinaesthetic Experience , 2017, Conference on Designing Interactive Systems.

[44]  Katherine Isbister,et al.  Making Sense of Human-Food Interaction , 2019, CHI.

[45]  C. Spence,et al.  Multisensory Technology for Flavor Augmentation: A Mini Review , 2018, Front. Psychol..

[46]  Weisi Lin,et al.  Mulsemedia: State of the Art, Perspectives, and Challenges , 2014, TOMM.

[47]  D. Senkowski,et al.  The multifaceted interplay between attention and multisensory integration , 2010, Trends in Cognitive Sciences.

[48]  Takuji Narumi,et al.  The future of computing and food: extended abstract , 2018, AVI.

[49]  Jas Brooks,et al.  Trigeminal-based Temperature Illusions , 2020, CHI.