Proceedings of the 1st Workshop on Multi-sensorial Approaches to Human-Food Interaction

Eating and drinking are, perhaps, some of the most multisensory events of our everyday life. Take, for instance, flavor, which is one of the most important elements of such experiences. It is known that flavor is the product of the integration of, at least, gustatory and (retronasal) olfactory cues. Nevertheless, researchers have suggested that all our senses can influence the way in which we perceive flavor, not to mention our eating and drinking experiences. For instance, the color and shape of the food, the background sonic cues in our eating environments, and/or the sounds that derive from the food's mastication can all influence our perception and enjoyment of our eating and drinking experiences. In this workshop, we were particularly interested in new systems that were designed to enhance people's eating experiences in the context of HFI and which were based on the principles that govern the systematic connections that exist between the senses (e.g., spatiotemporal congruence, semantic congruence, and crossmodal correspondences. This included the experiencing food interactions digitally in remote locations, sensing flavor information from one place, transferring them over the internet digitally, and effectively regenerate at the destination. Further, we were interested in digital interfaces that would bring advantages such as precious controlling, cheaper maintenance, avoid refilling, and avoid calories. Therefore, in this workshop we called for studies on flavor sensing and actuation interfaces, new communication mediums, and persisting and retrieving technologies for HFI. Enhancing social interactions to augment the eating experience was another issue we intended addressed in this workshop. In addition, we wanted to discuss what is possible through multimodal technology and what is not possible without it during this workshop. Factors such as measurement techniques (e.g. mastication, eating speed, food tracking, psychophysiological responses to food consumption), potential for interactivity, and potential for customized experiences were taken into consideration. Finally, applications of multisensory approaches to HFI were also encouraged to submit since they can promote healthy eating habits, design of food-related products (e.g. packaging) and more compelling eating experiences.

[1]  Abdellah Touhafi,et al.  TASTE - Testing Auditory Solutions Towards the Improvement of the Tasting Experience , 2013 .

[2]  Abdellah Touhafi,et al.  Wireless Sensor Networks for Sound Design: A Summary on Possibilities and Challenges , 2016 .

[3]  Martin R. Yeomans,et al.  The role of expectancy in sensory and hedonic evaluation: The case of smoked salmon ice-cream , 2008 .

[4]  Marc O Ernst,et al.  Natural auditory scene statistics shapes human spatial hearing , 2014, Proceedings of the National Academy of Sciences.

[5]  J. Verhagen,et al.  The neurocognitive bases of human multimodal food perception: Sensory integration , 2006, Neuroscience & Biobehavioral Reviews.

[6]  John M. de Castro,et al.  Listening to music while eating is related to increases in people's food intake and meal duration , 2006, Appetite.

[7]  Ole G Mouritsen,et al.  The emerging science of gastrophysics and its application to the algal cuisine , 2012, Flavour.

[8]  Barry C. Smith,et al.  The Nature of Sensory Experience: the Case of Taste and Tasting , 2013 .

[9]  Casey O'Callaghan,et al.  SEEING WHAT YOU HEAR: CROSS‐MODAL ILLUSIONS AND PERCEPTION , 2008 .

[10]  Charles Spence,et al.  Multisensory flavour perception , 2013, Current Biology.

[11]  M. Haverkamp,et al.  mayo 2015 CAN SYNESTHETIC PERCEPTION HELP TO DEFINE ATTRACTIVE PRODUCT DESIGN ? , 2015 .

[12]  Charles Spence,et al.  Sensory expectations elicited by the sounds of opening the packaging and pouring a beverage , 2015, Flavour.

[13]  Masahiko Inami,et al.  Straw-Like User Interface (II): A New Method of Presenting Auditory Sensations for a More Natural Experience , 2008, EuroHaptics.

[14]  Koji Tsukada,et al.  Sensing fork: eating behavior detection utensil and mobile persuasive game , 2013, CHI Extended Abstracts.

[15]  L. Marks,et al.  Effect of endogenous attention on detection of weak gustatory and olfactory flavors , 2004, Perception & psychophysics.

[16]  C. Spence Crossmodal correspondences: A tutorial review , 2011, Attention, perception & psychophysics.

[17]  Thomas Hummel,et al.  Investigations on multimodal sensory integration: Texture, taste, and ortho- and retronasal olfactory stimuli in concert , 2007, Neuroscience Letters.

[18]  R. Stevenson,et al.  Flavor binding: Its nature and cause. , 2014, Psychological bulletin.

[19]  Zohar Eitan,et al.  How music touches: Musical parameters and listeners’ audio-tactile metaphorical mappings , 2011 .

[20]  Carmel A Levitan,et al.  Assessing the role of color cues and people's beliefs about color-flavor associations on the discrimination of the flavor of sugar-coated chocolates. , 2008, Chemical senses.

[21]  Michael Haverkamp,et al.  Synesthetic approach for evaluation of the cross-sensory quality of multi-media applications , 2010, 2010 Second International Workshop on Quality of Multimedia Experience (QoMEX).

[22]  I. Lundström,et al.  An electronic tongue based on voltammetry , 1997 .

[23]  Aleksandra Mroczko-Wąsowicz Perception-Cognition Interface & Cross-Modal Experiences: Insights into Unified Consciousness , 2017 .

[24]  Per B. Brockhoff,et al.  Feelings as a basis for discrimination: Comparison of a modified authenticity test with the same–different test for slightly different types of milk , 2007 .

[25]  C. Spence,et al.  Crossmodal correspondences between sounds and tastes , 2012, Psychonomic bulletin & review.

[26]  Charles Spence,et al.  Assessing the Role of Sound in the Perception of Food and Drink , 2010 .

[27]  M. Cabanac,et al.  Influence of noise on gustatory affective ratings and preference for sweet or salt , 1987, Appetite.

[28]  Amy C. Traylor,et al.  What Virtual Reality Research in Addictions Can Tell Us about the Future of Obesity Assessment and Treatment , 2011, Journal of diabetes science and technology.

[29]  Juyun Lim,et al.  Potential mechanisms of retronasal odor referral to the mouth. , 2011, Chemical senses.

[30]  Carlos Velasco,et al.  Music Influences Hedonic and Taste Ratings in Beer , 2016, Front. Psychol..

[31]  T. Hummel,et al.  Cross-modal integration between odors and abstract symbols , 2010, Neuroscience Letters.

[32]  J. Delwiche The impact of perceptual interactions on perceived flavor , 2004 .

[33]  Marcia Grabowecky,et al.  Object-based auditory facilitation of visual search for pictures and words with frequent and rare targets. , 2011, Acta psychologica.

[34]  C. Spence,et al.  Colour, pleasantness, and consumption behaviour within a meal , 2014, Appetite.

[35]  C. Spence Multisensory Flavor Perception , 2015, Cell.

[36]  Robin Dando,et al.  A crossmodal role for audition in taste perception. , 2015, Journal of experimental psychology. Human perception and performance.

[37]  Lawrence L. Garber,et al.  Placing Food Color Experimentation into a Valid Consumer Context , 2001 .

[38]  C. Spence,et al.  A bittersweet symphony: Systematically modulating the taste of food by changing the sonic properties of the soundtrack playing in the background , 2012 .

[39]  Kate Wong,et al.  Neurogastronomy: How the brain creates flavor and why it matters. , 2011 .

[40]  M. Bourne,et al.  CRISPNESS IN FOODS-A REVIEW , 1976 .

[41]  Charles Spence,et al.  The role of auditory cues in modulating the perceived crispness and staleness of potato chips , 2004 .

[42]  Cesare Parise,et al.  Audiovisual Cross-Modal Correspondences in the General Population , 2013 .

[43]  L. Stafford,et al.  Effects of noise and distraction on alcohol perception , 2012 .

[44]  Geraint Rees,et al.  Sound alters activity in human V1 in association with illusory visual perception , 2006, NeuroImage.

[45]  Aleksandra Mroczko-Wasowicz,et al.  The unity and complexity of flavour perception: multisensory phenomenology and processing in human-food interactions , 2016, MHFI@ICMI.

[46]  Abdellah Touhafi,et al.  Using sound-taste correspondences to enhance the subjective value of tasting experiences , 2015, Front. Psychol..

[47]  C. Spence,et al.  The multisensory perception of flavor , 2008, Consciousness and Cognition.

[48]  C. Spence Eating with our ears: assessing the importance of the sounds of consumption on our perception and enjoyment of multisensory flavour experiences , 2015, Flavour.

[49]  R. E. Milliman,et al.  The Influence of Background Music on the Behavior of Restaurant Patrons , 1986 .

[50]  C. Spence,et al.  Does Food Color Influence Taste and Flavor Perception in Humans? , 2010 .

[51]  Robert Hurling,et al.  Eating with your eyes: effect of appearance on expectations of liking , 2003, Appetite.

[52]  Marc Leman,et al.  Does Music Influence the Multisensory Tasting Experience , 2015 .

[53]  Ali Israr,et al.  Touch, Taste, & Smell User Interfaces: The Future of Multisensory HCI , 2016, CHI Extended Abstracts.

[54]  Lina Engelen,et al.  Food Oral Processing: Fundamentals of Eating and Sensory Perception , 2012 .

[55]  Geke D. S. Ludden,et al.  Surprise and humor in product design: Designing sensory metaphors in multiple modalities , 2012 .

[56]  Ophelia Deroy The Unity Assumption and the Many Unities of Consciousness , 2014 .

[57]  John Zimmerman,et al.  Your dinner's calling: supporting family dinnertime activities , 2007, DPPI.

[58]  Jan O. Borchers,et al.  First-person cooking: a dual-perspective interactive kitchen counter , 2010, CHI EA '10.

[59]  S. Fiske,et al.  The Structure of Value: Accounting for Taste George Mandler , 2014 .

[60]  Adrian David Cheok,et al.  Chop Chop: A Sound Augmented Kitchen Prototype , 2012, Advances in Computer Entertainment.

[61]  Xuan Wang,et al.  CoDine: an interactive multi-sensory system for remote dining , 2011, UbiComp '11.

[62]  Abdellah Touhafi,et al.  Assessing multisensory tasting experiences by means of customized soundscapes , 2015 .

[63]  Michael Haverkamp,et al.  Synesthetic Design: Handbook for a Multi-Sensory Approach , 2012 .

[64]  Hidekazu Tanaka,et al.  Chewing jockey: augmented food texture by using sound based on the cross-modal effect , 2011, SA '11.

[65]  Ann Heylighen,et al.  Designing spaces for every listener , 2010, Universal Access in the Information Society.

[66]  R. Stevenson,et al.  The role of taste and oral somatosensation in olfactory localization , 2011, Quarterly journal of experimental psychology.

[67]  C. Spence,et al.  Smelling shapes: crossmodal correspondences between odors and shapes. , 2013, Chemical senses.

[68]  Karen B. Schloss,et al.  Music-to-Color Associations of Single-Line Piano Melodies in Non-synesthetes. , 2016, Multisensory research.

[69]  Charles Spence,et al.  Grape expectations: The role of cognitive influences in color–flavor interactions , 2010, Consciousness and Cognition.

[70]  Katsumi Takahashi,et al.  Human interface using PC display with head pointing device for eating assist robot and emotional evaluation by GSR sensor , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[71]  C. Spence,et al.  The Perfect Meal: The Multisensory Science of Food and Dining , 2014 .

[72]  L. Marks The Unity of the Senses: Interrelations Among the Modalities , 1978 .

[73]  C. Spence,et al.  Crossmodal Space and Crossmodal Attention , 2004 .

[74]  Charles Spence,et al.  Oral referral: On the mislocalization of odours to the mouth , 2016 .

[75]  M. Wiesmann,et al.  Emotional stimulation alters olfactory sensitivity and odor judgment. , 2007, Chemical senses.

[76]  Charles Spence,et al.  “Smooth operator”: Music modulates the perceived creaminess, sweetness, and bitterness of chocolate , 2017, Appetite.

[77]  N. Block On a confusion about a function of consciousness , 1995, Behavioral and Brain Sciences.

[78]  C. Spence,et al.  The influence of auditory cues on the perception of, and responses to, food and drink , 2010 .

[79]  Tim Bayne The multisensory nature of perceptual consciousness , 2014 .

[80]  Homei Miyashita,et al.  Communication by change in taste , 2011, CHI Extended Abstracts.

[81]  C. Spence,et al.  Crossmodal correspondences between odors and contingent features: odors, musical notes, and geometrical shapes , 2013, Psychonomic bulletin & review.

[82]  Adrian David Cheok,et al.  Eating with our eyes: From visual hunger to digital satiation , 2016, Brain and Cognition.

[83]  Akio Yamamoto,et al.  Straw-like user interface: virtual experience of the sensation of drinking using a straw , 2006, ACE '06.

[84]  D. Small Flavor is in the brain , 2012, Physiology & Behavior.

[85]  Michael Haverkamp,et al.  Synesthetic Design – Building Multi-sensory Arrangements , 2009 .

[86]  John Prescott,et al.  Multisensory processes in flavour perception and their influence on food choice , 2015 .

[87]  R. Stevenson,et al.  The role of attention in the localization of odors to the mouth , 2011, Attention, perception & psychophysics.

[88]  Adrian David Cheok,et al.  Sensory marketing in light of new technologies , 2015, Advances in Computer Entertainment.

[89]  C. Spence,et al.  The multisensory perception of flavor: Assessing the influence of color cues on flavor discrimination responses , 2007 .

[90]  Nicolas Guéguen,et al.  Sound level of environmental music and drinking behavior: a field experiment with beer drinkers. , 2008, Alcoholism, clinical and experimental research.

[91]  Felipe Reinoso Carvalho,et al.  Matching soundscapes and music with food types , 2016 .

[92]  Charles Spence,et al.  Airplane noise and the taste of umami , 2014, Flavour.

[93]  Abdellah Touhafi,et al.  ABox: New method for evaluating wireless acoustic-sensor networks , 2014 .

[94]  Gijs Huisman,et al.  A Moving Feast: Effects of Color, Shape and Animation on Taste Associations and Taste Perceptions , 2016, ACE.

[95]  Adrian David Cheok,et al.  Crossmodal correspondences between taste and shape, and their implications for product packaging: A review , 2016 .

[96]  Koji Tsukada,et al.  EducaTableware: computer-augmented tableware to enhance the eating experiences , 2013, CHI Extended Abstracts.

[97]  J. Vroomen,et al.  Intersensory binding across space and time: A tutorial review , 2013, Attention, Perception, & Psychophysics.

[98]  Juyun Lim,et al.  The role of congruency in retronasal odor referral to the mouth. , 2012, Chemical senses.

[99]  Wen Jiun Yap,et al.  Design and development of Multi-touchable E-restaurant Management System , 2010, 2010 International Conference on Science and Social Research (CSSR 2010).

[100]  Qian Wang,et al.  The influence of soundscapes on the perception and evaluation of beers , 2016 .

[101]  Carlos Velasco,et al.  The Sound of Temperature: What Information do Pouring Sounds Convey Concerning the Temperature of a Beverage , 2013 .

[102]  C. Spence,et al.  Hedonic mediation of the crossmodal correspondence between taste and shape , 2015 .

[103]  Charles Spence,et al.  Colour-taste correspondences: Designing food experiences to meet expectations or to surprise , 2016 .

[104]  Kenton O'Hara,et al.  FoodWorks: tackling fussy eating by digitally augmenting children's meals , 2014, NordiCHI.

[105]  Daniel Shepherd,et al.  Listening to music can influence hedonic and sensory perceptions of gelati , 2016, Appetite.

[106]  Lawrence E. Marks,et al.  Synesthesia: Strong and Weak , 2001 .

[107]  Charles Spence,et al.  Wine and music (II): can you taste the music? Modulating the experience of wine through music and sound , 2015, Flavour.

[108]  Charles Stangor,et al.  Social groups in action and interaction , 2004 .

[109]  Ellen Poliakoff,et al.  Effect of background noise on food perception , 2011 .

[110]  F. Marinetti The Founding and Manifesto of Futurism , 2016 .

[111]  Charles Spence,et al.  Auditory contributions to flavour perception and feeding behaviour , 2012, Physiology & Behavior.

[112]  Charles Spence,et al.  The taste of cutlery: how the taste of food is affected by the weight, size, shape, and colour of the cutlery used to eat it , 2013, Flavour.

[113]  Jun Rekimoto,et al.  Spot-Light: Multimodal Projection Mapping on Food , 2013, HCI.

[114]  C Spence,et al.  The Complex Interplay Between Multisensory Integration and Perceptual Awareness. , 2016, Multisensory research.

[115]  Geke D.S. Ludden,et al.  Surprise As a Design Strategy , 2008, Design Issues.

[116]  J. Melichar,et al.  Human Taste Thresholds Are Modulated by Serotonin and Noradrenaline , 2006, The Journal of Neuroscience.

[117]  Eric Robinson,et al.  Evaluation of a Smart Fork to Decelerate Eating Rate. , 2016, Journal of the Academy of Nutrition and Dietetics.

[118]  Wang Yi,et al.  AutoDietary: A Wearable Acoustic Sensor System for Food Intake Recognition in Daily Life , 2016, IEEE Sensors Journal.

[119]  John Prescott,et al.  Flavour as a psychological construct: implications for perceiving and measuring the sensory qualities of foods , 1999 .

[120]  J. Delwiche,et al.  You eat with your eyes first , 2012, Physiology & Behavior.

[121]  Ryan S. Elder,et al.  The crunch effect: Food sound salience as a consumption monitoring cue , 2016 .

[122]  田村 秀行,et al.  Mixed reality : merging real and virtual worlds , 1999 .

[123]  Tony Cheng,et al.  Sensory integration and the unity of consciousness , 2016 .

[124]  Constantine Stephanidis,et al.  Universal access in the information society , 1999, HCI.

[125]  Brie Gertler,et al.  Privileged access : philosophical accounts of self-knowledge , 2017 .

[126]  Geke D.S. Ludden,et al.  Effects of visual-auditory incongruity on product expression and surprise , 2007 .

[127]  Charles Spence,et al.  Noise and its impact on the perception of food and drink , 2014, Flavour.

[128]  Jeremy N. Bailenson,et al.  Virtual Experiences, Physical Behaviors: The Effect of Presence on Imitation of an Eating Avatar , 2009, PRESENCE: Teleoperators and Virtual Environments.

[129]  Larissa Hjorth,et al.  TastyBeats: Designing Palatable Representations of Physical Activity , 2015, CHI.

[130]  Abdellah Touhafi,et al.  Using Sound to Enhance Taste Experiences: An Overview , 2016, CMMR.

[131]  Ellen Yi-Luen Do,et al.  Simulating the sensation of taste for immersive experiences , 2013, ImmersiveMe '13.

[132]  John R. Piggott Understanding flavour quality: Difficult or impossible? , 1994 .