Synaesthesia and synergy in art. Gustav Mahler’s “Symphony No. 2 in C minor” as an example of interactive music visualization

Synaesthesia occurs when stimulation of one sensory modality induces an experience in another sensory modality. This phenomenon can involuntarily emerge or be intentionally induced by artists through the synthesis of image or color and music. Consequently in the present chapter the author differentiates between the emergence of synaesthesia, understood as a neurological condition characterized by exceptional sensory perception and investigated in medicine, psychology, and recently in the neurosciences, and a long-standing tradition of synaesthetic oeuvres in the history and theory of art and culture.

[1]  S. Baron-Cohen,et al.  Hearing Words and Seeing Colours: An Experimental Investigation of a Case of Synaesthesia , 1987, Perception.

[2]  Manfred Clynes,et al.  Music, Mind, and Brain , 1982, Springer US.

[3]  J. M. Baker,et al.  The Music of Alexander Scriabin , 1986 .

[4]  M. Clynes,et al.  The Living Quality of Music , 1982 .

[5]  Wassily Kandinsky,et al.  Concerning the Spiritual in Art , 1977 .

[6]  F. Galton.  Inquiries into Human Faculty and Its Development , 1883 .

[7]  Richard E. Cytowic.  Synesthesia: A union of the senses, 2nd ed. , 2002 .

[8]  Jamie Ward,et al.  Sound-Colour Synaesthesia: to What Extent Does it Use Cross-Modal Mechanisms Common to us All? , 2006, Cortex.

[9]  Daniel Smilek,et al.  The Role of Meaning in Grapheme-Colour Synaesthesia , 2006, Cortex.

[10]  R. Melara,et al.  Dimensional interaction between color and pitch. , 1989, Journal of experimental psychology. Human perception and performance.

[11]  Michael Haverkamp.  Synästhetisches Design: Kreative Produktentwicklung für alle Sinne , 2008 .

[12]  Isaac Sir Newton.  Opticks, or, A treatise of the reflections, refractions, inflections & colours of light , 1933 .

[13]  Daniel Smilek,et al.  Not all synaesthetes are created equal: Projector versus associator synaesthetes , 2004, Cognitive, affective & behavioral neuroscience.

[14]  Jamie Ward,et al.  Non-random associations of graphemes to colours in synaesthetic and non-synaesthetic populations , 2005, Cognitive neuropsychology.

[15]  E. T. Bullmore,et al.  Functional magnetic resonance imaging of synesthesia: activation of V4/V8 by spoken words , 2002, Nature Neuroscience.

[16]  Michael Haverkamp,et al.  LOOK AT THAT SOUND! VISUAL ASPECTS OF AUDITORY PERCEPTION , 2009 .

[17]  A. Liberman,et al.  The motor theory of speech perception revised , 1985, Cognition.

[18]  F. Galton.  I.—STATISTICS OF MENTAL IMAGERY , 1880 .

[19]  S. Baron-Cohen,et al.  Coloured Speech Perception: Is Synaesthesia what Happens when Modularity Breaks Down? , 1993, Perception.

[20]  Julia Simner,et al.  Beyond perception: synaesthesia as a psycholinguistic phenomenon , 2007, Trends in Cognitive Sciences.

[21]  V. Ramachandran,et al.  Synaesthesia? A window into perception, thought and language , 2001 .

[22]  Edward M. Hubbard,et al.  Neurocognitive Mechanisms of Synesthesia , 2005, Neuron.

[23]  Kenneth J. Peacock,et al.  Instruments to Perform Color-Music: Two Centuries of Technological Experimentation , 2017 .

[24]  Simon Baron-Cohen,et al.  Synaesthesia : classic and contemporary readings , 1997 .

[25]  Newton on the number of colours in the spectrum , 1990 .

[26]  Christopher T. Lovelace,et al.  Mechanisms of synesthesia: cognitive and physiological constraints , 2001, Trends in Cognitive Sciences.

[27]  Simon Baron-Cohen,et al.  Synaesthesia: Prevalence and Familiality , 1996, Perception.

[28]  Daniel Smilek,et al.  The Importance of Individual Differences in Grapheme-Color Synesthesia , 2005, Neuron.

[29]  F. Marinetti.  The Futurist Cookbook , 1989 .

[30]  G. Stocker.  Hybrid : Living in Paradox , 2005 .

[31]  D. Ricco.  Sinestesie per il design. Le interazioni sensoriali nell'epoca dei multimedia , 1999 .

[32]  Nicholas Cook,et al.  Music, Imagination, And Culture , 1991 .

[33]  C. Baudelaire.  The Flowers of Evil , 1982 .

[34]  Klaus-Ernst Behne.  Film, Musik, Video, oder, Die Konkurrenz von Auge und Ohr , 1987 .

[35]  Des Phenomènes de Synopsie , 1893 .

[36]  D. Eagleman,et al.  A standardized test battery for the study of synesthesia , 2007, Journal of Neuroscience Methods.

[37]  N. Todd.  The dynamics of dynamics: A model of musical expression , 1992 .

[38]  Julia Simner,et al.  Synaesthetic visuo-spatial forms: Viewing sequences in space , 2009, Cortex.

[39]  J. Harrison,et al.  Synaesthesia: An Account of Coloured Hearing , 1994 .

[40]  Sean M. Smith,et al.  A visualization of music , 1997, Proceedings. Visualization '97 (Cat. No. 97CB36155).

[41]  Jean-Baptiste Cazier,et al.  A whole-genome scan and fine-mapping linkage study of auditory-visual synesthesia reveals evidence of linkage to chromosomes 2q24, 5q33, 6p12, and 12p12. , 2009, American journal of human genetics.

[42]  Jorg Jewanski.  Ist C = Rot? Eine Kultur- und Wissenschaftsgeschichte zum Problem der wechselseitigen Beziehung zwischen Ton und Farbe Von Aristoteles bis Goethe , 1999 .

[43]  Georg Anschütz.  Farbe - Ton - Forschungen , 1927 .

[44]  V. Ramachandran,et al.  Psychophysical investigations into the neural basis of synaesthesia , 2001, Proceedings of the Royal Society of London. Series B: Biological Sciences.

[45]  Julian E. Asher,et al.  Diagnosing and Phenotyping Visual Synaesthesia: a Preliminary Evaluation of the Revised Test of Genuineness (TOG-R) , 2006, Cortex.

[46]  J. Simner,et al.  Early detection of markers for synaesthesia in childhood populations. , 2009, Brain : a journal of neurology.

[47]  Diatoms in the London Clay , 1880, Nature.

[48]  Stefan Strötgen.  Michael Haverkamp, Synästhetisches Design. Kreative Produktentwicklung für alle Sinne, München, Wien: Hanser 2009 , 2012 .

[49]  Jamie Ward,et al.  Synaesthesia: The taste of words on the tip of the tongue , 2006, Nature.

[50]  Gerfried Stocker,et al.  Simplicity : the art of complexity , 2006 .

[51]  Alexander Truslit.  Gestaltung und Bewegung in der Musik , 1938 .

[52]  D. R. Hay.  The Laws of Harmonious Colouring: Adapted to Interior Decorations, Manufactures, and Other Useful Purposes , 2013 .

[53]  Elisa Bertino,et al.  A Semantic-Based Execution Model for Multilevel Secure Workflows , 2000, J. Comput. Secur..

[54]  Jean Paul,et al.  Vorschule der Ästhetik , 1990 .

[55]  D. Eagleman,et al.  Wednesday Is Indigo Blue: Discovering the Brain of Synesthesia , 2009 .

[56]  Fiona N. Newell,et al.  Synaesthesia is associated with enhanced, self-rated visual imagery , 2008, Consciousness and Cognition.