Throwing, Pitching and Catching Sound: Audio Windowing Models and Modes

Abstract After surveying the concepts of audio windowing, this paper elaborates taxonomies of three sets of its dimensions—spatial audio ("throwing sound"), timbre ("pitching sound"), and gain ("catching sound")—establishing matrices of variability for each, drawing similes, and citing applications. Two audio windowing systems are examined across these three operations: repositioning, distortion/blending, and gain control (i.e. state transitions in virtual space, timbre space, and volume space). Handy Sound is a purely auditory system with gestural control, while MAW exploits exocentric graphical control. These two systems motivated the development of special user interface features. (Sonic) piggyback-channels are introduced as filtear manifestations of changing cursors, used to track control state. A variable control/response ratio can be used to map a near-field work envelope into perceptual space. Clusters can be used to hierarchically collapse groups of spatial sound objects. WIMP idioms are reinterpreted for audio windowing functions. Reflexive operations are cast an instance of general manipulation when all the modified entities, including an iconification of the user, are projected into an egalitarian control/response system. Other taxonomies include a spectrum of directness of manipulation, and sensitivity to current position crossed with dependency on some target position.

[1]  Pierre Boulez,et al.  Computers in Music , 1988 .

[2]  Elizabeth M. Wenzel,et al.  Localization in Virtual Acoustic Displays , 1992, Presence: Teleoperators & Virtual Environments.

[3]  William W. Gaver.  The SonicFinder: An Interface That Uses Auditory Icons , 1989, Hum. Comput. Interact..

[4]  John M. Chowning,et al.  THE SIMULATION OF MOVING SOUND SOURCES , 1970 .

[5]  Gary S. Kendall.  Visualization by Ear: Auditory Imagery for Scientific Visualization and Virtual Reality , 1991 .

[6]  Sara Bly,et al.  Presenting information in sound , 1982, CHI '82.

[7]  William Buxton,et al.  Communicating with sound , 1987 .

[8]  J M Loomis,et al.  Active localization of virtual sounds. , 1990, The Journal of the Acoustical Society of America.

[9]  Scott H. Foster,et al.  A Virtual Display System for Conveying Three-Dimensional Acoustic Information , 1988 .

[10]  Michael Cohen,et al.  Extending the notion of a window system to audio , 1990, Computer.

[11]  William L. Martens,et al.  Simulating the Cues of Spatial Hearing in Natural Environments , 1984, ICMC.

[12]  Alain Martel,et al.  The SS-1 Sound Spatializer: A Real-Time MIDI Spatialization Processor , 1986, ICMC.

[13]  William W. Gaver,et al.  Effective sounds in complex systems: the ARKOLA simulation , 1991, CHI.

[14]  Douglas Scott.  A Processor for Locating Stationary and Moving Sound Sources in a Simulated Acoustical Environment , 1989, ICMC.

[15]  J. Gibson.  The Ecological Approach to Visual Perception , 1979 .

[16]  William W. Gaver.  Auditory Icons: Using Sound in Computer Interfaces , 1986, Hum. Comput. Interact..

[17]  Richard L. McKinley,et al.  Digital synthesis of binaural auditory localization azimuth cues using headphones , 1988 .

[18]  Daniel G. Bobrow,et al.  Beyond the chalkboard: computer support for collaboration and problem solving in meetings , 1988, CACM.

[19]  Gary S. Kendall,et al.  Spatial reverberation: discussion and demonstration , 1989 .

[20]  R. Daniel Bergeron,et al.  Stereophonic and surface sound generation for exploratory data analysis , 1990, CHI '90.

[21]  Elizabeth M. Wenzel,et al.  Virtual Interface Environment Workstations , 1988 .

[22]  E. C. Cherry.  Some Experiments on the Recognition of Speech, with One and with Two Ears , 1953 .

[23]  D R Begault,et al.  Preferred Sound Intensity Increase for Sensation of Half Distance , 1991, Perceptual and motor skills.

[24]  Frederic L. Wightman,et al.  An Exploratory Study of the Use of Movement-Correlated Cues in an Auditory Head-Up Display , 1989 .

[25]  A. D. N. Edwards,et al.  The design of auditory interfaces for visually disabled users , 1988, CHI '88.

[26]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[27]  Diane H. Sonnenwald,et al.  InfoSound: an audio aid to program comprehension , 1990, Twenty-Third Annual Hawaii International Conference on System Sciences.

[28]  Alistair D. N. Edwards,et al.  Modelling Blind Users' Interactions with an Auditory Computer Interface , 1989, Int. J. Man Mach. Stud..

[29]  Scott S. Fisher,et al.  A system for three-dimensional acoustic 'visualization' in a virtual environment workstation , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[30]  Keith A. Lantz,et al.  An experiment in integrated multimedia conferencing , 1986, CSCW '86.

[31]  Warren Robinett,et al.  Virtual environment display system , 1987, I3D '86.

[32]  Michael Benedikt,et al.  Cyberspace: First Steps , 1991 .

[33]  R. M. Greenberg,et al.  Guidelines for the syntactic design of audio cues in computer interfaces , 1985 .

[34]  Daniel G. Bobrow,et al.  WYSIWIS revised: early experiences with multi-user interfaces , 1986, CSCW.

[35]  Sunil K. Sarin,et al.  Computer-Based Real-Time Conferencing Systems , 1985, Computer.

[36]  Kenneth L. Kraemer,et al.  Computer-based systems for cooperative work and group decisionmaking: status of use and problems in development , 1986, CSCW '86.

[37]  E. J. Addeo,et al.  An experimental multi-media bridging system , 1988 .

[38]  Michael Cohen,et al.  Multidimensional Audio Window Management , 1991, Int. J. Man Mach. Stud..

[39]  Nicola Bernardini,et al.  TRAILS: An Interactive System for Sound Location , 1989, ICMC.

[40]  I. Pollack,et al.  The Information of Elementary Multidimensional Auditory Displays , 1954 .

[41]  Mark S. Sanders,et al.  Human Factors in Engineering and Design , 2016 .

[42]  William L. Martens,et al.  A Spatial Sound Processor for Loudspeaker and Headphone Reproduction , 1990 .

[43]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[44]  Stephen Wilson,et al.  The Media Lab , 1987 .

[45]  Michael Cohen.  Multidimensional audio windows: extending user interfaces through the use of spatial auditory information , 1991 .

[46]  Meera M. Blattner,et al.  Communicating and Learning Through Non-speech Audio , 1992 .

[47]  Robert C. Morrison,et al.  A Microcomputer-Based Laboratory Aid for Visually Impaired Students , 1983, IEEE Micro.

[48]  William L. Martens,et al.  Image Model Reverberation from Recirculating Delays , 1986 .

[49]  J. Foley.  Interfaces for advanced computing , 1987 .

[50]  J. Mezrich,et al.  Dynamic Representation of Multivariate Time Series Data , 1984 .