Multisensory Technology for Flavor Augmentation: A Mini Review

There is growing interest in the development of new technologies that capitalize on our emerging understanding of the multisensory influences on flavor perception in order to enhance human–food interaction design. This review focuses on the role of (extrinsic) visual, auditory, and haptic/tactile elements in modulating flavor perception and more generally, our food and drink experiences. We review some of the most exciting examples of recent multisensory technologies for augmenting such experiences. Here, we discuss applications for these technologies, for example, in the field of food experience design, in the support of healthy eating, and in the rapidly growing world of sensory marketing. However, as the review makes clear, while there are many opportunities for novel human–food interaction design, there are also a number of challenges that will need to be tackled before new technologies can be meaningfully integrated into our everyday food and drink experiences.

[1]  W. Sato,et al.  Fat Content Modulates Rapid Detection of Food: A Visual Search Study Using Fast Food and Japanese Diet , 2017, Front. Psychol..

[2]  C. Spence.  Multisensory Flavor Perception , 2015, Cell.

[3]  C. Spence,et al.  Technology at the dining table , 2013, Flavour.

[4]  Yasushi Ikei,et al.  High-Resolution Tactile Display for Lips , 2016, HCI.

[5]  C. Spence,et al.  A bittersweet symphony: Systematically modulating the taste of food by changing the sonic properties of the soundtrack playing in the background , 2012 .

[6]  Gijs Huisman,et al.  A Moving Feast: Effects of Color, Shape and Animation on Taste Associations and Taste Perceptions , 2016, ACE.

[7]  Charles Spence,et al.  Plateware and slurping influence regular consumers׳ sensory discriminative and hedonic responses to a hot soup , 2017 .

[8]  Chi Thanh Vi,et al.  Not just seeing, but also feeling art: Mid-air haptic experiences integrated in a multisensory art exhibition , 2017, Int. J. Hum. Comput. Stud..

[9]  Daniel Saakes,et al.  “See me, feel me”: Effects of 3D-printed surface patterns on beverage evaluation , 2017 .

[10]  Charles Spence,et al.  Sensory expectations based on product-extrinsic food cues: An interdisciplinary review of the empirical evidence and theoretical accounts , 2015 .

[11]  C. Spence.  Eating with our ears: assessing the importance of the sounds of consumption on our perception and enjoyment of multisensory flavour experiences , 2015, Flavour.

[12]  Chi Thanh Vi,et al.  Multisensory Experiences in HCI , 2017, IEEE MultiMedia.

[13]  Martin V. Butz,et al.  Behavioral Bias for Food Reflected in Hand Movements: A Preliminary Study with Healthy Subjects , 2016, Cyberpsychology Behav. Soc. Netw..

[14]  Takamichi Nakamoto,et al.  Olfactory display using surface acoustic wave device and micropumps for wearable applications , 2016, 2016 IEEE Virtual Reality (VR).

[15]  Yuta Sugiura,et al.  Gravitamine spice: a system that changes the perception of eating through virtual weight sensation , 2015, AH.

[16]  Michael Haverkamp,et al.  Synesthetic Design: Handbook for a Multi-Sensory Approach , 2012 .

[17]  Charles Spence,et al.  Haptic exploration of plateware alters the perceived texture and taste of food , 2016 .

[18]  Ellen Poliakoff,et al.  Effect of background noise on food perception , 2011 .

[19]  Takuji Narumi,et al.  Meta Cookie+: An Illusion-Based Gustatory Display , 2011, HCI.

[20]  Gijs Huisman,et al.  1st international workshop on multi-sensorial approaches to human-food interaction (workshop summary) , 2016, ICMI.

[21]  Robin Dando,et al.  A crossmodal role for audition in taste perception. , 2015, Journal of experimental psychology. Human perception and performance.

[22]  Amit Zoran,et al.  Cornucopia: The Concept of Digital Gastronomy , 2011, Leonardo.

[23]  Adrian David Cheok,et al.  Sensory marketing in light of new technologies , 2015, Advances in Computer Entertainment.

[24]  Carlos Velasco,et al.  Multisensory Experiences , 2020 .

[25]  C. Spence,et al.  Cutlery matters: heavy cutlery enhances diners’ enjoyment of the food served in a realistic dining environment , 2015, Flavour.

[26]  Koji Tsukada,et al.  EducaTableware: computer-augmented tableware to enhance the eating experiences , 2013, CHI Extended Abstracts.

[27]  Ellen Yi-Luen Do,et al.  FunRasa: an interactive drinking platform , 2014, TEI '14.

[28]  Adrian David Cheok,et al.  Can Food Porn Make Us Slim? How Brains of Consumers React to Food in Digital Environments , 2016 .

[29]  N. Stroebele,et al.  Effect of ambience on food intake and food choice. , 2004, Nutrition.

[30]  Hidekazu Tanaka,et al.  Chewing jockey: augmented food texture by using sound based on the cross-modal effect , 2011, SA '11.

[31]  Akio Yamamoto,et al.  Straw-like user interface: virtual experience of the sensation of drinking using a straw , 2006, ACE '06.

[32]  Takuji Narumi,et al.  Visual-olfactory display using olfactory sensory map , 2010, 2010 IEEE Virtual Reality Conference (VR).

[33]  Katsunori Okajima,et al.  Projective-AR system for customizing the appearance and taste of food , 2016, MVAR '16.

[34]  Hiroo Iwata,et al.  Food simulator: a haptic interface for biting , 2004, IEEE Virtual Reality 2004.

[35]  Marcus Foth,et al.  Eat, Cook, Grow: Mixing Human-Computer Interactions with Human-Food Interactions , 2014 .

[36]  Xuan Wang,et al.  CoDine: an interactive multi-sensory system for remote dining , 2011, UbiComp '11.

[37]  Takuji Narumi,et al.  Taste of breath: the temporal order of taste and smell synchronized with breathing as a determinant for taste and olfactory integration , 2017, Scientific Reports.

[38]  Chinatsu Kasamatsu,et al.  Cognitive structures based on culinary success factors in the development of new dishes by Japanese chefs at fine dining restaurants , 2015, Flavour.

[39]  K. Tsukada,et al.  EaTheremin , 2011, SA '11.

[40]  Ryuta Okazaki,et al.  Change in the Amount Poured as a Result of Vibration When Pouring a Liquid , 2014, AsiaHaptics.

[41]  Adrian David Cheok,et al.  Crossmodal correspondences between taste and shape, and their implications for product packaging: A review , 2016 .

[42]  Maureen Morrin,et al.  Does Touch Affect Taste? The Perceptual Transfer of Product Container Haptic Cues , 2008 .

[43]  Ali Israr,et al.  Sensing the future of HCI , 2016, Interactions.

[44]  J. Verhagen,et al.  The neurocognitive bases of human multimodal food perception: Sensory integration , 2006, Neuroscience & Biobehavioral Reviews.

[45]  B. G. Slocombe,et al.  Cross-modal tactile–taste interactions in food evaluations , 2016, Neuropsychologia.

[46]  Gijs Huisman,et al.  Tasty tech: human-food interaction and multimodal interfaces , 2016, MHFI@ICMI.

[47]  Damien Blackwell,et al.  Gastrophysics. The new science of eating [Book Review] , 2017 .

[48]  Charles Spence,et al.  The role of auditory cues in modulating the perceived crispness and staleness of potato chips , 2004 .

[49]  Hendrik N.J. Schifferstein,et al.  Tools Facilitating Multi-sensory Product Design , 2008 .

[50]  Bertil Hultén,et al.  What is Sensory Marketing , 2009 .

[51]  Ryan S. Elder,et al.  The crunch effect: Food sound salience as a consumption monitoring cue , 2016 .

[52]  Jun Rekimoto,et al.  Spot-Light: Multimodal Projection Mapping on Food , 2013, HCI.

[53]  John Prescott,et al.  Multisensory processes in flavour perception and their influence on food choice , 2015 .

[54]  Adrian David Cheok,et al.  Implementing new food interactions using magnetic dining table platform and magnetic foods , 2016, MVAR '16.

[55]  C. Spence,et al.  Effects of Visual Texture on Food Perception , 2013 .

[56]  Waka Fujisaki,et al.  The effect of a crunchy pseudo-chewing sound on perceived texture of softened foods , 2016, Physiology & Behavior.

[57]  Adrian David Cheok,et al.  Eating with our eyes: From visual hunger to digital satiation , 2016, Brain and Cognition.

[58]  Carl Machover,et al.  Virtual reality , 1994, IEEE Computer Graphics and Applications.

[59]  Richard Harper,et al.  Celebratory technology: new directions for food research in HCI , 2008, CHI.

[60]  C. Levitan,et al.  Red hot: the crossmodal effect of color intensity on perceived piquancy. , 2014, Multisensory research.

[61]  Anton Nijholt,et al.  A multisensory approach for the design of food and drink enhancing sonic systems , 2016, MHFI@ICMI.

[62]  C. Spence,et al.  A large sample study on the influence of the multisensory environment on the wine drinking experience , 2014, Flavour.

[63]  Takuji Narumi,et al.  Multi-sensorial virtual reality and augmented human food interaction , 2016, MHFI@ICMI.

[64]  Daniel Shepherd,et al.  Listening to music can influence hedonic and sensory perceptions of gelati , 2016, Appetite.

[65]  Kenton O'Hara,et al.  Designing for human – food interaction: An introduction to the special issue on ‘ food and interaction design ’ , 2013 .

[66]  Katsunori Okajima,et al.  Effects of Visual Food Texture on Taste Perception , 2011 .

[67]  Carlos Velasco,et al.  Searching for flavor labels in food products: the influence of color-flavor congruence and association strength , 2015, Front. Psychol..

[68]  Carlos Velasco,et al.  Digitizing the chemical senses: Possibilities & pitfalls , 2017, Int. J. Hum. Comput. Stud..

[69]  Ryuta Okazaki,et al.  Audio-Haptic Rendering of Water Being Poured from Sake Bottle , 2013, Advances in Computer Entertainment.

[70]  Brenda K. Wiederhold,et al.  Virtual Reality in the Assessment and Treatment of Weight-Related Disorders , 2016, Cyberpsychology Behav. Soc. Netw..