Smell-O-Message: Integration of Olfactory Notifications into a Messaging Application to Improve Users' Performance

Smell is a powerful tool for conveying and recalling information without requiring visual attention. Previous work identified, however, some challenges caused by user's unfamiliarity with this modality and complexity in the scent delivery. We are now able to overcome these challenges, introducing a training approach to familiarise scent-meaning associations (urgency of a message, and sender identity) and using a controllable device for the scent-delivery. Here we re-validate the effectiveness of smell as notification modality and present findings on the performance of smell in conveying information. In a user study composed of two sessions, we compared the effectiveness of visual, olfactory, and combined visual-olfactory notifications in a messaging application. We demonstrated that olfactory notifications improve users' confidence and performance in identifying the urgency level of a message, with the same reaction time and disruption levels as for visual notifications. We discuss the design implications and opportunities for future work in the domain of multimodal interactions.

[1]  Yulin Wang,et al.  Designing wearable vibrotactile notifications for information communication , 2016, Int. J. Hum. Comput. Stud..

[2]  Eamonn O'Neill,et al.  Auditory icon and earcon mobile service notifications: intuitiveness, learnability, memorability and preference , 2009, CHI.

[3]  Ellen Yi-Luen Do,et al.  Vocktail: A Virtual Cocktail for Pairing Digital Taste, Smell, and Color Sensations , 2017, ACM Multimedia.

[4]  Karon E. MacLean,et al.  Exploiting haptic facets: Users' sensemaking schemas as a path to design and personalization of experience , 2017, Int. J. Hum. Comput. Stud..

[5]  T. Hongratanaworakit Relaxing Effect of Rose Oil on Humans , 2009, Natural product communications.

[6]  David K. McGookin,et al.  Hajukone: Developing an Open Source Olfactory Device , 2016, CHI Extended Abstracts.

[7]  Enrico Rukzio,et al.  inScent: a wearable olfactory display as an amplification for mobile notifications , 2017, SEMWEB.

[8]  Judith Amores,et al.  Essence: Olfactory Interfaces for Unconscious Influence of Mood and Cognitive Performance , 2017, International Conference on Human Factors in Computing Systems.

[9]  Eric Horvitz,et al.  Learning and reasoning about interruption , 2003, ICMI '03.

[10]  Brian P. Bailey,et al.  If not now, when?: the effects of interruption at different moments within task execution , 2004, CHI.

[11]  Takuji Narumi,et al.  Augmented reality flavors: gustatory display based on edible marker and cross-modal interaction , 2011, CHI.

[12]  G. Norman Likert scales, levels of measurement and the “laws” of statistics , 2010, Advances in health sciences education : theory and practice.

[13]  Fredrik U. Jönsson,et al.  Olfactory working memory: effects of verbalization on the 2-back task , 2011, Memory & cognition.

[14]  Andreas Keller,et al.  The evolutionary function of conscious information processing is revealed by its task-dependency in the olfactory system , 2014, Front. Psychol..

[15]  J D Gabrieli,et al.  Sniffing longer rather than stronger to maintain olfactory detection threshold. , 2000, Chemical senses.

[16]  V. Parma,et al.  Processing of Human Body Odors , 2017 .

[17]  J. McGann,et al.  Poor human olfaction is a 19th-century myth , 2017, Science.

[18]  Chi Thanh Vi,et al.  What Did I Sniff?: Mapping Scents Onto Driving-Related Messages , 2017, AutomotiveUI.

[19]  Takamichi Nakamoto,et al.  Cooking Up an Interactive Olfactory Game Display , 2008, IEEE Computer Graphics and Applications.

[20]  Joseph Nathaniel Kaye,et al.  Symbolic olfactory display , 2001 .

[21]  J. Andrade,et al.  EVIDENCE FOR AN OLFACTORY STORE IN WORKING MEMORY , 2007 .

[22]  Stephen A. Brewster,et al.  The Role of Modality in Notification Performance , 2011, INTERACT.

[23]  Tao Zhang,et al.  Smelling directions: Olfaction modulates ambiguous visual motion perception , 2014, Scientific Reports.

[24]  D. Broadbent,et al.  What makes interruptions disruptive? A study of length, similarity, and complexity , 1989 .

[25]  Ellen Yi-Luen Do,et al.  Season Traveller: Multisensory Narration for Enhancing the Virtual Reality Experience , 2018, CHI.

[26]  A. Keller,et al.  Attention and Olfactory Consciousness , 2011, Front. Psychology.

[27]  David Warnock,et al.  A subjective evaluation of multimodal notifications , 2011, 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[28]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[29]  Philip Kortum,et al.  HCI Beyond the GUI: Design for Haptic, Speech, Olfactory, and Other Nontraditional Interfaces , 2008 .

[30]  A. Dittmar,et al.  Basic emotions induced by odorants: a new approach based on autonomic pattern results. , 1999, Journal of the autonomic nervous system.

[31]  M. Bensafi,et al.  Psychophysiological correlates of affects in human olfaction , 2002, Neurophysiologie Clinique/Clinical Neurophysiology.

[32]  Stanley Lemeshow,et al.  Olfactory influences on mood and autonomic, endocrine, and immune function , 2008, Psychoneuroendocrinology.

[33]  Mary Czerwinski,et al.  Notification, Disruption, and Memory: Effects of Messaging Interruptions on Memory and Performance , 2001, INTERACT.

[34]  Shinjiro Kawato,et al.  Projection based olfactory display with nose tracking , 2004, IEEE Virtual Reality 2004.

[35]  Ryan P. McMahan,et al.  Development of a Simple and Low-Cost Olfactory Display for Immersive Media Experiences , 2014, ImmersiveMe '14.

[36]  Richard Corbett,et al.  AROMA: ambient awareness through olfaction in a messaging application , 2004, ICMI '04.

[37]  Alexandre N. Tuch,et al.  Opportunities for odor: experiences with smell and implications for technology , 2014, CHI.

[38]  R. Boakes,et al.  Changes in Odor Sweetness Resulting from Implicit Learning of a Simultaneous Odor-Sweetness Association: An Example of Learned Synesthesia ☆ ☆☆ , 1998 .

[39]  M. L. Demattè,et al.  Olfactory discrimination: when vision matters? , 2008, Chemical senses.

[40]  Ernesto Arroyo,et al.  Interruptions as multimodal outputs: which are the less disruptive? , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[41]  David M. Frohlich,et al.  Timespace in the workplace: dealing with interruptions , 1995, CHI 95 Conference Companion.

[42]  Stephen A. Brewster,et al.  The impact of unwanted multimodal notifications , 2011, ICMI '11.

[43]  K. Durand,et al.  The Nose Tells it to the Eyes: Crossmodal Associations between Olfaction and Vision , 2010, Perception.

[44]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[45]  T. Hummel,et al.  Effects of olfactory training in patients with olfactory loss , 2009, The Laryngoscope.

[46]  T. Hummel,et al.  Olfactory training is helpful in postinfectious olfactory loss: A randomized, controlled, multicenter study , 2014, The Laryngoscope.

[47]  Mei-Kei Lai,et al.  Universal scent blackbox: engaging visitors communication through creating olfactory experience at art museum , 2015, SIGDOC.

[48]  Joseph Kaye,et al.  Making Scents: aromatic output for HCI , 2004, INTR.

[49]  Christopher Miller,et al.  Olfoto: designing a smell-based interaction , 2006, CHI.

[50]  Shin Yamamoto,et al.  Study on Stimulation Effects for Driver Based on Fragrance Presentation , 2011, MVA.

[51]  P. Lachenbruch Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[52]  Daniël Lakens,et al.  Calculating and reporting effect sizes to facilitate cumulative science: a practical primer for t-tests and ANOVAs , 2013, Front. Psychol..

[53]  Leslie M Kay,et al.  Olfactory Coding: Random Scents Make Sense , 2011, Current Biology.

[54]  A. Maravita,et al.  Smelling the Space Around Us: Odor Pleasantness Shifts Visuospatial Attention in Humans , 2017, Emotion.

[55]  Umberto Cugini,et al.  VR system for rehabilitation based on hand gestural and olfactory interaction , 2015, VRST.

[56]  Susan C. Roberts,et al.  Variability of affective responses to odors: culture, gender, and olfactory knowledge. , 2013, Chemical senses.

[57]  T. Engen,et al.  Associations to odors: interference, mnemonics, and verbal labeling. , 1977, Journal of experimental psychology. Human learning and memory.

[58]  T. White,et al.  Incongruent Contextual Information Intrudes on Short-term Olfactory Memory , 2011 .

[59]  Marc Hassenzahl,et al.  Everything can be beautiful , 2012, INTR.

[60]  D. G. Laing,et al.  Influence of training and experience on the perception of multicomponent odor mixtures. , 1996, Journal of experimental psychology. Human perception and performance.

[61]  Chi Thanh Vi,et al.  A Comparison of Scent-Delivery Devices and Their Meaningful Use for In-Car Olfactory Interaction , 2016, AutomotiveUI.

[62]  Edgar Erdfelder,et al.  G*Power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences , 2007, Behavior research methods.

[63]  Ken-ichi Okada,et al.  System for Presenting and Creating Smell Effects to Video , 2014, ICMI.

[64]  R. Mehta,et al.  Blue or Red? Exploring the Effect of Color on Cognitive Task Performances , 2009, Science.

[65]  Albrecht Schmidt,et al.  Implicit human computer interaction through context , 2000, Personal Technologies.

[66]  C. Spence,et al.  Auditory perceptual load: A review , 2017, Hearing Research.

[67]  Juan Liu,et al.  Multi-sensory interaction thechnology and its system application , 2010 .

[68]  Maliha S. Nash,et al.  Handbook of Parametric and Nonparametric Statistical Procedures , 2001, Technometrics.

[69]  S. Nordin,et al.  A Scandinavian Adaptation of the Multi-Clinic Smell and Taste Questionnaire: Evaluation of Questions About Olfaction , 2003, Acta oto-laryngologica.

[70]  Marianna Obrist,et al.  OSpace: Towards a Systematic Exploration of Olfactory Interaction Spaces , 2017, ISS.

[71]  N. Pedersen,et al.  Odor identification: influences of age, gender, cognition, and personality. , 2000, The journals of gerontology. Series B, Psychological sciences and social sciences.

[72]  David Warnock,et al.  The application of multiple modalities for improved home care reminders , 2012, CHI Extended Abstracts.

[73]  Fahim Kawsar,et al.  The myth of subtle notifications , 2014, UbiComp Adjunct.

[74]  S. Firestein How the olfactory system makes sense of scents , 2001, Nature.

[75]  Stephen A. Brewster,et al.  Multiple notification modalities and older users , 2013, CHI.

[76]  Daniel C. McFarlane,et al.  Comparison of Four Primary Methods for Coordinating the Interruption of People in Human-Computer Interaction , 2002, Hum. Comput. Interact..

[77]  Kara A. Latorella,et al.  The Scope and Importance of Human Interruption in Human-Computer Interaction Design , 2002, Hum. Comput. Interact..