The Sound of the Smell of my Shoes

Given the sensory poverty of virtual environments, such as those found in computer games that rely, in the main, solely on audio-visual interfaces, how best do we attain the experience of presence in those environments when presence requires the construction of a coherent (in the sense of realism) place in which to be and in which to act? The paper explores this question through an investigation of the senses of hearing and smell and suggests the possibility of introducing the experience of odours into such environments through the use of sound.

[1]  A. Moore.  The spectacular anus of Joseph Pujol: Recovering the Pétomane’s unique historic context , 2013 .

[2]  Joseph M. Williams.  Synaesthetic Adjectives: A Possible Law of Semantic Change , 1976 .

[3]  Paul A. Cairns,et al.  A grounded investigation of game immersion , 2004, CHI EA '04.

[4]  G. J. Train,et al.  Three Essays on The Theory of Sexuality , 1966 .

[5]  S. Kirby,et al.  What Sound Does That Taste? Cross-Modal Mappings across Gustation and Audition , 2010, Perception.

[6]  Cristina Cacciari.  The Cambridge Handbook of Metaphor and Thought: Crossing the senses in metaphorical language , 2008 .

[7]  Gordon Calleja,et al.  Immersion in Virtual Worlds , 2014 .

[8]  M. Smeets,et al.  Proust revisited: Odours as triggers of aversive memories , 2012, Cognition & emotion.

[9]  Paul A. Cairns,et al.  Measuring and defining the experience of immersion in games , 2008, Int. J. Hum. Comput. Stud..

[10]  G. Lakoff,et al.  Women, Fire, and Dangerous Things: What Categories Reveal about the Mind , 1988 .

[11]  Mel Slater,et al.  A note on presence terminology , 2003 .

[12]  Wijnand A. IJsselsteijn,et al.  Presence: Where Are We? , 2001, Cyberpsychology Behav. Soc. Netw..

[13]  Mandayam A. Srinivasan,et al.  The Effect of Auditory Cues on the Haptic Perception of Stiffness in Virtual Environments , 1997, Dynamic Systems and Control.

[14]  A. Gilbert,et al.  Auditory Pitch as a Perceptual Analogue to Odor Quality , 1997 .

[15]  E. Heuberger,et al.  Do ambient urban odors evoke basic emotions? , 2013, Front. Psychol..

[16]  R. Bowen Loftin,et al.  MULTIMODAL DISPLAY SYSTEMS : HAPTIC , OLFACTORY , GUSTATORY , AND VESTIBULAR , 2008 .

[17]  E. Köster,et al.  A “Misfit” Theory of Spontaneous Conscious Odor Perception (MITSCOP): reflections on the role and function of odor memory in everyday life , 2014, Front. Psychol..

[18]  F. Tomita,et al.  Foot odor due to microbial metabolism and its control. , 2006, Canadian journal of microbiology.

[19]  Olfaction, Taste, and Cognition: Nose-wise: Olfactory Metaphors in Mind , 2002 .

[20]  Mark Grimshaw,et al.  Imagining sound , 2014, Audio Mostly Conference.

[21]  Yoshitaka Nakajima,et al.  Auditory Scene Analysis: The Perceptual Organization of Sound Albert S. Bregman , 1992 .

[22]  Karen Collins,et al.  Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design , 2008 .

[23]  M. Grimshaw,et al.  Game Sound Technology and Player Interaction: Concepts and Developments , 2010 .

[24]  Mark Grimshaw,et al.  Sound and Player Immersion in Digital Games , 2011 .

[25]  Donald A. Wilson,et al.  Smelling Sounds: Olfactory–Auditory Sensory Convergence in the Olfactory Tubercle , 2010, The Journal of Neuroscience.

[26]  R. Holm.  Challenge of pain. , 2008, South Dakota medicine : the journal of the South Dakota State Medical Association.

[27]  Rudi Giot,et al.  Volatiles that Sound Bad: Sonification of Defensive Chemical Signals from Insects against Insects , 2014 .

[28]  H. Jaseja.  Scientific basis behind traditional practice of application of “shoe-smell” in controlling epileptic seizures in the eastern countries , 2008, Clinical Neurology and Neurosurgery.

[29]  L. Marks.  The Unity of the Senses , 1978 .

[30]  Alan Chalmers.  Level of Realism , 2014 .

[31]  A. Clark.  Being There: Putting Brain, Body, and World Together Again , 1996 .

[32]  N. Presmeg.  The body in the mind: The bodily basis of meaning, imagination and reason , 1992 .

[33]  Robert J. Zatorre,et al.  There’s more to auditory cortex than meets the ear , 2007, Hearing Research.

[34]  Mark Grimshaw,et al.  Sonic Virtuality: Sound as Emergent Perception , 2015 .

[35]  Alan Chalmers,et al.  Real virtuality: a step change from virtual reality , 2009, SCCG.

[36]  R. Gibbs.  The Cambridge Handbook of Metaphor and Thought , 2008 .

[37]  W. Takken,et al.  Composition of Human Skin Microbiota Affects Attractiveness to Malaria Mosquitoes , 2011, PloS one.

[38]  Karl Koschutnig,et al.  Successful Contextual Integration of Loose Mental Associations As Evidenced by Emotional Conflict-Processing , 2014, PloS one.

[39]  Jason B. Castro,et al.  Olfaction, valuation, and action: reorienting perception , 2014, Front. Psychol..

[40]  G. Lakoff.  Women, fire, and dangerous things : what categories reveal about the mind , 1989 .

[41]  Lawrence M. Zbikowski.  Conceptualizing Music: Cognitive Structure, Theory, and Analysis , 2002 .

[42]  Malcolm Riddoch.  On the non-Cochlearity of the sounds Themselves , 2012, ICMC.

[43]  R. Calkin,et al.  Perfumery: Practice and Principles , 1994 .

[44]  Mark Grimshaw,et al.  Sound And Immersion In The First-Person Shooter , 2007, Int. J. Intell. Games Simul..

[45]  R. Hudson,et al.  Olfaction, Taste, and Cognition: The Individuality of Odor Perception , 2002 .

[46]  Klaus Frieler,et al.  What is the Sound of Citrus? Research on the Correspondences between the Percept ion of Sound and Flavour , 2012 .

[47]  P. Varela,et al.  Playing with Sound , 2013 .

[48]  C. Spence,et al.  Crossmodal correspondences between sounds and tastes , 2012, Psychonomic bulletin & review.