We describe three systems that use natural or event-based sounds as means of data delivery. In these systems we have mapped data to natural sounds using metaphors. In the first system we evaluate the use of sounds of air, horn, and train to convey ordered numeric values between 1 to 6. An example of the metaphor used here is the association of speed values to the sound of a moving train at different speeds. In the second system, we use sounds of ocean waves to convey whether the exposure in a protein structural alignment is buried, partially exposed or fully exposed. The metaphor used here is the association of sound with how exposed the user is with respect to the ocean. In the third system, we map animal sounds such as the sound of a roaring lion or a chirping bird to certain stocks based on user preferences. The behavior of the stocks are then sounded by the use of whistles and car crash to signify the movement in process of the stocks. An up whistling sound can be clearly associated with an uptrend. We present and discuss the results of user evaluation studies for all the three systems.

Visualization researchers of the Exploratory Visualization (Exvis) project are studying the representation of multidimensional databases as two-dimensional arrays of data-driven icons. Each data point in n-dimensional space is converted into one icon in the display; both visual and auditory features of the icon are determined by the data. The display's texture is produced by packing large numbers of small icons together so densely that they lose their individual identities. The premise of the technology is that interesting features in the visual and auditory texture of an iconographic display will point to interesting features in the data. The technology is in the early stages of formal study. The short-term goal is to provide a workstation that will enable a researcher who is neither a programmer nor a trained experimentalist to design, implement, conduct and analyze human factors experiments for studying the iconographic data-display technique. The long-term goal is to use the information gathered from such experiments to provide a powerful data-representation language for scientists to use for visualizing large data sets.<<ETX>>

Visual and aural portrayals of parallel program execution are used to gain insight into how a program is working. The combination of portrayals in a coordinated performance environment provides the user with multiple perspectives and stimuli to comprehend complex, multidimensional run-time information. An open question for either medium is how well does it scale? That is, how effectively can it be used to represent program performance on a large parallel computer system? This paper investigates using sound in conjunction with graphics to represent the performance of a scalable application program, the SLALOM benchmark program, executed on the nCUBE 2 distributed memory parallel computer. Custom auralization software is coupled with the PICL and ParaGraph tools. The techniques and results of visually and aurally monitoring program execution on increasing numbers of processors are presented.<<ETX>>

Virtual auditory space - context, acoustics and psychoacoustics overview of spatial hearing azimuth and elevation perception sound source distance and environmental context implementing 3-D sound application examples what's next ? - future and futuristic 3-D sound applications.

Virtual Auditory Space technology is being applied to a broad range of applications in a wide variety of fields. The flexibility and fine degree of stimulus control enabled by VAS techniques makes them perfectly suited for studying sound localization and spatial auditory perception. Researchers can easily separate out the relative importance of different spatial cues, manipulate stimulus “sources” with ease and speed, and generate stimuli which are difficult or impossible to generate using free-field techniques. This same level of flexibility and control makes VAS displays useful in providing spatial auditory information to a human operator in many real-world applications. In the past, visual displays have been employed almost exclusively to present spatial information to air traffic controllers, fighter pilots, and other human operators. However, there is a growing need to increase the amount of information received by such operators. VAS techniques provide the opportunity to exploit the auditory channel without directly compromising the information already available via other modalities. VAS displays are also integral parts of multimodal “virtual environment” (VE) systems (proposed for use in training and entertainment) and “teleoperator” systems (designed to display information from remote or dangerous environments).

Verification is a crucial process to facilitate the identification and removal of errors within simulations. This study explores semantic changes to the concept of simulation verification over the past six decades using a data-supported, automated content analysis approach. We collect and utilize a corpus of 4,047 peer-reviewed Modeling and Simulation (M&S) publications dealing with a wide range of studies of simulation verification from 1963 to 2015. We group the selected papers by decade of publication to provide insights and explore the corpus from four perspectives: (i) the positioning of prominent concepts across the corpus as a whole; (ii) a comparison of the prominence of verification, validation, and Verification and Validation (V&V) as separate concepts; (iii) the positioning of the concepts specifically associated with verification; and (iv) an evaluation of verification’s defining characteristics within each decade. Our analysis reveals unique characterizations of verification in each decade. The insights gathered helped to identify and discuss three categories of verification challenges as avenues of future research, awareness, and understanding for researchers, students, and practitioners. These categories include conveying confidence and maintaining ease of use; techniques’ coverage abilities for handling increasing simulation complexities; and new ways to provide error feedback to model users.

Up to now virtual reality (VR) systems have emphasized visual graphics and display technology. With audio hardware and system software readily available it is now possible to take the next evolutionary step and use acoustic simulations to enhance virtual worlds. An audiovisual system addresses two important human senses and provides realistic impressions that are natural to the daily life and environment. The VR toolkit of IGD features several visual and acoustic renderers, which are applied by IGD's general purpose VR system "Virtual Design". Examples demonstrate several virtual audiovisual worlds.<<ETX>>

Traditionally, visualization systems have focused on the visual sense. However, with the advent of multimedia and virtual reality systems, other senses such as sound and touch are being slowly incorporated into systems. Even in the visual channel, the majority of systems depend on the perception of geometry through graphical concepts such as lines, fillareas, windows, and raster pixmaps to provide the visualization feedback. Sound is being effectively used in visualization systems and is increasingly being integrated into mainstream systems. However, we have not made much progress in developing a fundamental understanding of interaction in non-geometric representational spaces. We are interested in extending simple interactions, such as zoom and pan, into other domains such as sound. Pitch is a perceptual quantity of sound that is associated with the physical quantity frequency. We describe how zoom and pan operations in pitch are supported. Formal definitions for these operations are also provided. Finally, we describe a prototype system for such interactions.

To access interactive systems, blind and visually impaired users can leverage their auditory senses by using non-speech sounds. The current structure of non-speech sounds, however, is geared toward conveying user interface operations (e.g., opening a file) rather than large theme-based information (e.g., a history passage) and, thus, is ill-suited to signify the complex meanings of primary learning material (e.g., books and websites). In order to address this problem, this dissertation introduces audemes, a new category of non-speech sounds, whose semiotic structure and flexibility open new horizons for facilitating the education of blind and visually impaired students. An experiment with 21 students from the Indiana School for the Blind and Visually Impaired (ISBVI) supports the hypothesis that audemes increase the retention of theme-based information. By acting as memory catalysts, audemes can play an important role in enhancing the aural interaction and navigation in future sound-based user interfaces. For this dissertation, I designed an Acoustic EDutainment INterface (AEDIN) that integrates audemes as a way by which to vividly anticipate text-to-speech theme-based information and, thus, act as innovative aural covers. The results of two iterative usability evaluations with total of 20 blind and visually impaired participants showed that AEDIN is a highly usable and enjoyable acoustic interface. Yet, designing well-formed audemes remains an ad hoc process because audeme creators can only rely on their intuition to generate meaningful and memorable sounds. In order to address this problem, this dissertation presents three experiments, each with 10 blind and visually impaired participants. The goal was to examine the optimal combination of audeme attributes, which can be used to facilitate accurate recognitions of audeme meanings. This work led to the creation of seven basic guidelines that can be used to design well-formed audemes. An interactive application tool (ASCOLTA: Advanced Support and Creation-Oriented Library Tool for Audemes) operationalized these guidelines to support individuals without an audio background in designing well-formed audemes. An informal evaluation conducted with three teachers from the ISBVI, supports the hypothesis that ASCOLTA is a useful tool by which to facilitate the integration of audemes into the teaching environment.

Three alternative sonifications of proteomic data distributions were compared as a means to indicate the neuropathology associated with Amyotrophic Lateral Sclerosis (ALS) via auditory display (through exploration of the differentiation of induced pluripotent stem cell derived neurons). Pure visual displays of proteomic data often result in ”visual overload” such that detailed or subtle data important to describe ALS neurodegradation may be glossed over, and so three competing approaches to the sonification of proteomic data were designed to capitalize upon human auditory capacities that complement the visual capacities engaged by more conventional graphic representations. The auditory displays resulting from hypothesis-driven design of three alternative sonifications were evaluated by naïve listeners, who were instructed to listen for differences between the sonifications produce from proteomic data associated with three different types of cells. One of the sonifications was based upon the hypothesis that auditory sensitivity to regularities and irregularities in spatio-temporal patterns in the data could be heard through spatial distribution of sonification components. The design of a second sonification was based upon the hypothesis that variation in timbral components might create a distinguishable sound for each of three types of cells. A third sonification was based upon the hypothesis that redundant variation in both spatial and timbral components would be even more powerful as a means for identifying spatio-temporal patterns in the dynamic, multidimensional data generated in current proteomic studies of ALS.

This thesis examines the use of music for communicating information on the Human-Computer Interface. The current use of auditory signals (including the limited use of music) is reviewed and the perceptual advantages and drawbacks inherent in the use of music are identified. [Continues.]

This paper presents an analysis of progress in the use of sound as a tool in support of visualisation and gives an insight into its development and future needs. Special emphasis is given to the use of sound in scientific and engineering applications. A system developed to support surface data presentation and interaction by using sound is presented and discussed.

This paper presents a number of different sonification approaches that aim to communicate geometrical data, specifically curve shape and curvature information, of virtual 3-D objects. The system described here is part of a multi-modal augmented reality environment in which users interact with virtual models through the modalities vision, hearing and touch. An experiment designed to assess the performance of the sonification strategies is described and the key findings are presented and discussed.

This paper describes an auditory display being developed for a workstation designed to facilitate the visualization of multidimensional data. To establish the context of this work, the paper first discusses the psychophysical basis for using sound to represent multidimensional data, surveys the related literature, and describes the novel graphical technique employed by the workstation to display multidimensional data visually. Next, it gives an overview of the auditory display and its relationship to the visual display, emphasizing the roles played by the apparent location of sound sources and the repetition of sounds in time. Finally, the paper discusses the current prototype sound system and some of the difficulties to be faced in implementing the proposed facility. This work is being done as part of the Exploratory Visualization (Exvis) project at the University of Lowell.

This paper describes a study of the auditory environment in a chemical factory, and how a group of process operators ascribe meaning to a selection of sound clips from their daily work environment. We argue for a design-oriented phenomenological approach to soundscape studies, and suggest an approach based on an exploration of how already occurring sounds are used. This knowledge will be used to inform the design of new useful auditory environments. Our study shows that the richness of the auditory environment is a crucial aspect of the distributed work environment. An important part of the design process is the operator's contribution to the concepts suggested. From design workshops several design concepts aiming to explore and test different approaches for making sound affordances available have been developed. This has led us to a new understanding of how interactive soundscapes enable distributed awareness -- what we refer to as 'Being and Mixing'.

This paper addresses the question of how the work of the scientist will change in the new multimedia environments. Scenarios for the process of simulating and analyzing data in such environments are constructed, and some of the underlying models used in their construction are examined.<<ETX>>

This is a conference paper. The definitive version was published by Cambridge University Press (http://www.cambridge.org/uk/)

The run-time behavior of a program is defined by many parameters, thus it is frequently necessary for a programmer to consider the program's behavior from a number of different angles. This paper discusses the properties of parallel program behavior that, theoretically, can be mapped to sound, such that the natural properties of sound portray significant information about the program. Thus, when an aural display is combined with a graphical display, it will be possible for a user to gain more (or quicker) insight about a program's behavior than by using a graphical display alone. In support of this claim, this paper demonstrates the use of sound as an enhancement to graphical displays for a number of parallel programs. In particular the auralizations allow the listener to detect patterns within parallel program communication behavior and processor utilization distributions. A study of the uniqueness of these patterns, or signatures, is also discussed.<<ETX>>

The research explored how subjects in grade 1 (6-7 years old) and grade 3 (8-9 years old) identify auditory icons that are commonly introduced in educational software applications. The subjects were required to identify 37 auditory icons by specifying two dimensions: the object associated with producing the sound and the action which could be causing the sound. It was hypothesized that the ecological frequency versus relative uniqueness feature would be more salient for third-graders. Thus, they would be better equipped to identify the auditory cues than the first-graders. The results indicated that the third-grade subjects were better equipped to identify auditory cues based upon two dimensions of interest than the first-grade subjects. This information is useful for interface designers incorporating auditory icons into educational software for children at various developmental levels. Suggestions for future research are provided.

The representation of data in sound is emerging as a complement to data visualization. Several pilot studies over the past decade have proved the concept of auditory data representation; however, there has been little formal research to measure the effectiveness of auditory data representation techniques or to increase our understanding of how they work. Until quite recently, appropriate computing environments for research in this area simply did not exist. This situation is now improving and an increasing number of investigators are seeking solutions to the formidable problems posed by auditory data representation. This paper surveys the present state of the field and outlines the central problems which must be solved if the auditory data representation is to become a truly useful tool for data analysis and exploration.