Listener, Task, and Auditory Graph: Toward a Conceptual Model of Auditory Graph Comprehension

Auditory graph design and implementation often has been subject to criticisms of arbitrary or atheoretical decision-making processes in both research and application. Despite increasing interest in auditory displays coupled with more than two decades of auditory graph research, no theoretical models of how a listener processes an auditory graph have been proposed. The current paper seeks to present a conceptual level account of the factors relevant to the comprehension of auditory graphs by human listeners. We attempt to make links to the relevant literature on basic auditory perception, and we offer explicit justification for, or discussion of, a number of common design practices that are often justified only implicitly or by intuition in the auditory graph literature. Finally, we take initial steps toward a qualitative, conceptual level model of auditory graph comprehension that will help to organize the available data on auditory graph comprehension and make predictions for future research and applications with auditory graphs [

[1]  Lorna M. Brown,et al.  DESIGN GUIDELINES FOR AUDIO PRESENTATION OF GRAPHS AND TABLES , 2003 .

[2]  B. Burns Percepts, concepts, and categories : the representation and processing of information , 1992 .

[3]  Antti Pirhonen,et al.  NON-SPEECH SOUNDS AS ELEMENTS OF A USE SCENARIO: A SEMIOTIC PERSPECTIVE , 2006 .

[4]  C. Carswell,et al.  16 Reading Graphs: Interactions of Processing Requirements and Stimulus Structure , 1992 .

[5]  Bruce N. Walker,et al.  BRIEF TRAINING FOR PERFORMANCE OF A POINT ESTIMATION SONIFICATION TASK , 2005 .

[6]  Stephen Brewster,et al.  Using non-speech sound to overcome information overload , 1997 .

[7]  A Lewis,et al.  THE SCIENCE OF SOUND , 1997 .

[8]  Bruce N. Walker,et al.  PSYCHOPHYSICAL SCALING OF SONIFICATION MAPPINGS: A COMPARISION OF VISUALLY IMPAIRED AND SIGHTED LISTENERS , 2001 .

[9]  Bruce N. Walker,et al.  Effects of Auditory Context Cues and Training on Performance of a Point Estimation Sonification Task , 2005 .

[10]  W R GARNER The accuracy of counting repeated short tones. , 1951, Journal of experimental psychology.

[11]  Stephen A. Brewster,et al.  An evaluation of earcons for use in auditory human-computer interfaces , 1993, INTERCHI.

[12]  Ephraim P. Glinert,et al.  An experiment into the use of auditory cues to reduce visual workload , 1989, CHI '89.

[13]  Susan Bell Trickett,et al.  Toward a Comprehensive Model of Graph Comprehension: Making the Case for Spatial Cognition , 2006, Diagrams.

[14]  Mary Anne Buttigieg,et al.  Object Displays Do Not Always Support Better Integrated Task Performance , 1989 .

[15]  E. Schellenberg,et al.  Good Pitch Memory Is Widespread , 2003, Psychological science.

[16]  John H. Flowers,et al.  THIRTEEN YEARS OF REFLECTION ON AUDITORY GRAPHING: PROMISES, PITFALLS, AND POTENTIAL NEW DIRECTIONS , 2005 .

[17]  Lorna M. Brown,et al.  BROWSING MODES FOR EXPLORING SONIFIED LINE GRAPHS , 2002 .

[18]  Barbara Tversky,et al.  Graphs in Print , 2002, Diagrammatic Representation and Reasoning.

[19]  John G. Neuhoff,et al.  PITCH CHANGE, SONIFICATION, AND MUSICAL EXPERTISE: WHICH WAY IS UP? , 2002 .

[20]  Bruce N. Walker,et al.  Data density and trend reversals in auditory graphs: Effects on point-estimation and trend-identification tasks , 2008, TAP.

[21]  Gregory Kramer,et al.  An introduction to auditory display , 1994 .

[22]  Gregory Kramer,et al.  Pitch and loudness interact in auditory displays: can the data get lost in the map? , 2002, Journal of experimental psychology. Applied.

[23]  Priti Shah,et al.  A Model of the Perceptual and Conceptual Processes in Graph Comprehension , 1998 .

[24]  Terri L. Bonebright,et al.  Testing the effectiveness of sonified graphs for education: A programmatic research project , 2001 .

[25]  C T WHITE,et al.  Temporal numerosity. III. Auditory perception of number. , 1954, Journal of experimental psychology.

[26]  Stephen A. Brewster,et al.  Overcoming the Lack of Screen Space on Mobile Computers , 2002, Personal and Ubiquitous Computing.

[27]  D J Gillan,et al.  A Componential Model of Human Interaction with Graphs: 1. Linear Regression Modeling , 1994, Human factors.

[28]  D. Deutsch,et al.  The Internal Representation of Pitch Sequences in Tonal Music , 1981 .

[29]  Edward Childs Auditory graphs of real-time data , 2005 .

[30]  Priti Shah,et al.  Graph Comprehension: The Role of Format, Content and Individual Differences , 2002, Diagrammatic Representation and Reasoning.

[31]  Bruce N. Walker,et al.  Individual Differences, Cognitive Abilities, and the Interpretation of Auditory Graphs , 2004, ICAD.

[32]  Albert S. Bregman,et al.  Auditory scene analysis : hearing in complex environments , 1993 .

[33]  Bruce N. Walker,et al.  An agenda for research and development of multimodal graphs , 2005 .

[34]  W. Turnbull,et al.  Pitch discrimination as a function of tonal duration , 1944 .

[35]  Alberto de Campo,et al.  Toward a data sonification design space map , 2007 .

[36]  S. Kosslyn Understanding charts and graphs , 1989 .

[37]  A. Treisman,et al.  Emergent features, attention, and object perception. , 1984, Journal of experimental psychology. Human perception and performance.

[38]  W. Cleveland,et al.  Graphical Perception: Theory, Experimentation, and Application to the Development of Graphical Methods , 1984 .

[39]  Tony Stockman,et al.  AUDITORY GRAPHS: A SUMMARY OF CURRENT EXPERIENCE AND TOWARD S A RESEARCH AGENDA , 2005 .

[40]  John H. Flowers,et al.  Data sonification from the desktop: Should sound be part of standard data analysis software? , 2005, TAP.

[41]  M. Wertheimer Laws of organization in perceptual forms. , 1938 .

[42]  Stephen A. Brewster,et al.  Presenting dynamic information on mobile computers , 2006, Personal Technologies.

[43]  Steven Pinker,et al.  A theory of graph comprehension. , 1990 .

[44]  Jozsef A. Toth,et al.  The Role of Representation and Working Memory in Diagrammatic Reasoning and Decision Making , 2002, Diagrammatic Representation and Reasoning.

[45]  Susan Bell Trickett,et al.  A New Model of Graph and Visualization Usage , 2001 .

[46]  Gregory Kramer,et al.  Auditory Display: Sonification, Audification, And Auditory Interfaces , 1994 .

[47]  John H. Flowers,et al.  SONIFICATION OF DAILY WEATHER RECORDS: ISSUES OF PERCEPTION, ATTENTION AND MEMORY IN DESIGN CHOICES , 2001 .

[48]  Stephen Barrass,et al.  Auditory information design , 1998 .

[49]  John G. Neuhoff,et al.  Sonification Report: Status of the Field and Research Agenda Prepared for the National Science Foundation by members of the International Community for Auditory Display , 1999 .

[50]  Lorna M. Brown,et al.  DRAWING BY EAR: INTERPRETING SONIFIED LINE GRAPHS , 2003 .

[51]  Bruce N. Walker,et al.  Tick-marks, axes, and labels: The effects of adding context to auditory graphs , 2002 .

[52]  A Ehrenstein,et al.  Pitch and pitch change interact in auditory displays. , 2000, Journal of experimental psychology. Applied.

[53]  John H. Flowers,et al.  The effects of task demands on the equivalence of visual and auditory representations of periodic numerical data , 1996 .

[54]  S. S. Stevens Listening and Attention. , 1970 .

[55]  W. R. Garner,et al.  Pitch characteristics of short tones; pitch as a function of tonal duration. , 1948, Journal of experimental psychology.

[56]  Hesham M. Kamel,et al.  A Comparison of Three Nonvisual Methods for Presenting Scientific Graphs , 2002 .

[57]  Bruno L. Giordano,et al.  Spatial representation of pitch height: the SMARC effect , 2006, Cognition.

[58]  Judy Edworthy Does sound help us to work better with machines? A commentary on Rauterberg's paper ‘About the importance of auditory alarms during the operation of a plant simulator’ , 1998 .

[59]  Bruce N Walker,et al.  Magnitude estimation of conceptual data dimensions for use in sonification. , 2002, Journal of experimental psychology. Applied.

[60]  H B TIMMERMAN,et al.  What is task analysis? , 1951, Bulletin of the Medical Library Association.

[61]  David M. Lane,et al.  SONIFICATION OF ABSOLUTE VALUES WITH SINGLE AND MULTIPLE DIMENSIONS , 2003 .

[62]  Bruce N. Walker,et al.  RELATIVE INTENSITY OF AUDITORY CONTEXT FOR AUDITORY GRAPH DESIGN , 2006 .

[63]  M. R. Jones,et al.  Time, our lost dimension: toward a new theory of perception, attention, and memory. , 1976, Psychological review.

[64]  Charles S. Watson,et al.  Factors in the design of effective auditory displays , 1994 .

[65]  Robens Anderson,et al.  Creating an Empirical Framework for Sonification Design , 2005 .

[66]  Bruce N. Walker,et al.  Consistency of magnitude estimations with conceptual data dimensions used for sonification , 2007 .

[67]  Christopher Frauenberger,et al.  Patterns in auditory menu design , 2006 .

[68]  I. Nelken Demonstrations of Auditory Scene Analysis: The Perceptual Organization of Sound by Albert S. Bregman and Pierre A. Ahad, MIT Press, 1996. £15.95 CD , 1997, Trends in Neurosciences.