Sonifying Napoleon's March by Identifying Auditory Correlates of the Graphic-Linguistic Distinction

Identifying auditory correlates of the graphic-linguistic distinction informs our design of an auditory display based on Charles Minard’s depiction of Napoleon’s Russia campaign – the gold standard for visual (graphic) information design and therefore a grand challenge for auditory display design. We identify viable alternatives to the text-only translations currently employed in making graphics accessible to blind and/or low-vision individuals by introducing sounds bearing strong ecological resemblances to Minard’s depictions. Our integration of theoretical work about classic distinctions with common properties across diagrammatic and auditory display communities reveals practical opportunities for designing inclusive and accessible graphics.

[1]  Peter Coppin,et al.  What is Lost in Translation from Visual Graphics to Text for Accessibility , 2015, EAPCogSci.

[2]  John H. Flowers,et al.  Musical versus Visual Graphs: Cross-Modal Equivalence in Perception of Time Series Data , 1995, Hum. Factors.

[3]  Peter Keller,et al.  Meaning from environmental sounds: types of signal-referent relations and their effect on recognizing auditory icons. , 2004, Journal of experimental psychology. Applied.

[4]  Gregory Kramer,et al.  Pitch and loudness interact in auditory displays: can the data get lost in the map? , 2002, Journal of experimental psychology. Applied.

[5]  Gregory Kramer,et al.  An introduction to auditory display , 1994 .

[6]  Bruce N. Walker,et al.  Consistency of magnitude estimations with conceptual data dimensions used for sonification , 2007 .

[7]  P. Fayers,et al.  The Visual Display of Quantitative Information , 1990 .

[8]  Terri L. Bonebright,et al.  Testing the effectiveness of sonified graphs for education: A programmatic research project , 2001 .

[9]  Herbert A. Simon,et al.  Why a Diagram is (Sometimes) Worth Ten Thousand Words , 1987 .

[10]  J. Barwise,et al.  Visual information and valid reasoning , 1991 .

[11]  David A. Steinman,et al.  Progress toward sonifying Napoleon’s march and fluid flow simulations through binaural horizons , 2017 .

[12]  Mark C. Detweiler,et al.  Iconic Reference: Evolving Perspectives and an Organizing Framework , 1993, Int. J. Man Mach. Stud..

[13]  N. Goodman,et al.  Languages of art : an approach to a theory of symbols , 1979 .

[14]  Laurie M. Heller,et al.  ONE SMALL STEP: SOUND SOURCES AND EVENTS AS THE BASIS FOR AUDITORY GRAPHS , 2005 .

[15]  Peter Coppin,et al.  Iconic Properties are Lost when Translating Visual Graphics to Text for Accessibility , 2016 .

[16]  Atsushi Shimojima.  The Graphic-Linguistic DistinctionExploring Alternatives , 1999 .

[17]  Christopher R. Bennett,et al.  Touch-screen technology for the dynamic display of -2D spatial information without vision: promise and progress. , 2014, Multisensory research.

[18]  Daniel J. Levitin,et al.  Memory for musical attributes , 1999 .

[19]  Jaka Sodnik,et al.  Spatial Auditory Human-Computer Interfaces , 2015, SpringerBriefs in Computer Science.