LOOK AT THAT SOUND! VISUAL ASPECTS OF AUDITORY PERCEPTION

While it usually seems to be self-evident that perception of sound is a simple process, carried out within one sensory space, the existence of color hearing points at the fact that hearing includes visual aspects. In fact, not only during individual synesthetic processes, but even under common perceptual conditions sound refers to multi- sensory attributes. Thus, a multi-sensory design must be based on qualitative references between the senses. As an integral part, knowledge of the visual aspects of auditory perception is essential for creation of sounds which fit into a cross-modal environment. Various mechanisms of processing of stimuli ensure coupling of auditory qualia to multi-sensory perceptual objects. In the reality of daily life, even simple events which are heard include several references to those objects. Speech usually contains semantic and associative attributes as well as various analogies, like cross-sensory features (e.g. brightness, sharpness, volume), spatial distribution and temporal behavior (e.g. motion). Knowledge on spatial alignment of auditory, visual and tactile senses was inspired by research on spatial hearing capabilities and applied to design the multi-sensory environment surrounding human being. In this manner, a soundscape is the auditory analogy to a landscape, stimulating and reflected by the subjective emotionscape. As another aspect, correlations of dynamic noise, music and prosody of speech to visual motion are essential for perception of temporal changes of auditory events. Studies on the multi-sensory content of sound provide an in-sight into the complexity of auditory perception and illuminates the variety of tools available for multi-sensory approaches on sound design. Furthermore, genuine synesthetic experience offers a wide-ranging potential for advanced approaches.