Some Perspectives on Performed Sound and Music in Virtual Environments

The great variety of functions possible for sound in virtual environments is surveyed in relation to the traditions that primarily inform them. These traditions are examined, classifying sound into the three categories of artistic expression, information transfer, and environmental sound. The potentials of and relations between sonification, algorithmic composition, musicogenic and sonigenic displays, virtual musical instruments and virtual sound sources are examined, as well as the practical technical limitations that govern performance control of MIDI and real-time DSP sound synthesis in coordination with visual display. The importance of music-theoretic and psychological research is emphasized. The issues and developed categorizations are then applied to a case study: the examination of a specific virtual environment performance by a team of workers in Australia in which the author worked as composer/performer/ programmer.

[1]  M. Ashcraft.  Human memory and cognition , 1989 .

[2]  Michael Cohen,et al.  Throwing, Pitching and Catching Sound: Audio Windowing Models and Modes , 1993, Int. J. Man Mach. Stud..

[3]  Julius O. Smith,et al.  Physical Modeling Using Digital Waveguides , 1992 .

[4]  J. Morgan,et al.  A ROOM WITH A VIEW. , 1964, Canadian Medical Association journal.

[5]  Jeff Pressing.  Cybernetic Issues in Interactive Performance Systems , 1990 .

[6]  Michael Cohen,et al.  Integrating Graphic and Audio Windows , 1992, Presence: Teleoperators & Virtual Environments.

[7]  Ray Jackendoff,et al.  Musical Parsing and Musical Affect , 1991 .

[8]  Jeff Pressing,et al.  Nonlinear Maps as Generators of Musical Design , 1988 .

[9]  Leonard B. Meyer.  Emotion and Meaning in Music , 1957 .

[10]  Isabelle Peretz,et al.  Auditory agnosia: a functional analysis , 1993 .

[11]  Elizabeth M. Wenzel,et al.  Localization in Virtual Acoustic Displays , 1992, Presence: Teleoperators & Virtual Environments.

[12]  Daniel Thalmann.  Using Virtual Reality Techniques in the Animation Process , 1993, Virtual Reality Systems.

[13]  Jeff Pressing,et al.  The Micro- and Macrostructural Design of Improvised Music , 1987 .

[14]  John Orr.  Cinema and Modernity , 1993 .

[15]  Justine Sergent,et al.  Music, the brain and Ravel , 1993, Trends in Neurosciences.

[16]  Robert West,et al.  Representing musical structure , 1991 .

[17]  A. Small,et al.  Temporal differential sensitivity for auditory stimuli. , 1962, American Journal of Psychology.

[18]  Ivan E. Sutherland,et al.  The Ultimate Display , 1965 .

[19]  Michael A. Gigante.  Virtual Reality: Enabling Technologies , 1993, Virtual Reality Systems.

[20]  David Rosenboom,et al.  The Performing Brain , 1990 .

[21]  M. Schmuckler.  Expectation in music: Investigation of melodic and harmonic processes. , 1989 .

[22]  Meera Blattner,et al.  Earcons and Icons: Their Structure and Common Design Principles , 1989, Hum. Comput. Interact..

[23]  Gregory Kramer,et al.  Audification: The Use of Sound to Display Multivariate Data , 1991, ICMC.

[24]  Miller Puckette,et al.  Synthetic Rehearsal: Training the Synthetic Performer , 1985, ICMC.

[25]  B. Truax.  Acoustic Communication , 1985 .

[26]  D. M. Green,et al.  Sound localization by human listeners. , 1991, Annual review of psychology.

[27]  Pitoyo Hartono.  Adaptive timbre control using gesture , 1994 .

[28]  R. Daniel Bergeron,et al.  Stereophonic and surface sound generation for exploratory data analysis , 1990, CHI '90.

[29]  N Durlach.  Auditory Localization in Teleoperator and Virtual Environment Systems: Ideas, Issues, and Problems , 1991, Perception.

[30]  H J JERISON,et al.  Vigilance: A Review and Re-evaluation , 1963, Human factors.

[31]  Burton S. Rosner,et al.  The Perceptual Roles of Melodic Process, Contour, and Form , 1986 .

[32]  Alan B. Craig,et al.  Using sound to extract meaning from complex data , 1991, Electronic Imaging.

[33]  Gordon Monro.  Synthesis from attractors , 1993, ICMC.

[34]  Kevin Elliot.  A Behavioral or Actor-Based Paradigm of Sound Production and Studio Composition , 1993, ICMC.

[35]  Tapio Takala,et al.  Sound rendering , 1992, SIGGRAPH.

[36]  William W. Gaver.  The SonicFinder: An Interface That Uses Auditory Icons , 1989, Hum. Comput. Interact..

[37]  Joel Chadabe,et al.  Interactive Composing: An Overview , 1984, ICMC.

[38]  Hiroshi Akita,et al.  The Development of an Automatic Bagpipe Playing Device , 1993, ICMC.

[39]  Ruth Rasch,et al.  Synchronization in performed ensemble music , 1979 .

[40]  John C. Hart,et al.  The CAVE: audio visual experience automatic virtual environment , 1992, CACM.

[41]  Hiroshi Kubota,et al.  The Development of an Automatic Drum Playing Device , 1993, ICMC.

[42]  Scott H. Foster,et al.  Virtual environment display for a 3D audio room simulation , 1992, Electronic Imaging.

[43]  Camille Goudeseune,et al.  Model based interactive sound for an immersive virtual environment , 1994, ICMC.

[44]  J Edworthy,et al.  Improving Auditory Warning Design: Relationship between Warning Sound Parameters and Perceived Urgency , 1991, Human factors.

[45]  Alex Pentland,et al.  Device synchronization using an optimal linear filter , 1992, I3D '92.

[46]  M. Nyman.  Experimental Music: Cage and Beyond , 1974 .

[47]  R. Jackendoff,et al.  A Generative Theory of Tonal Music , 1985 .

[48]  Charles Dodge,et al.  Computer music , 1984 .

[49]  Sara Bly,et al.  Presenting information in sound , 1982, CHI '82.

[50]  Michael Cohen,et al.  Extending the notion of a window system to audio , 1990, Computer.

[51]  Jeff Pressing,et al.  Visualization and Predictive Modelling of Musical Signals using Embedding Techniques , 1993, ICMC.

[52]  Gary S. Kendall,et al.  An Implementation of a 3D Binaural Audio System within an Integrated Virtual Reality Environment , 1993, ICMC.

[53]  Brian Evans,et al.  Enhancing Scientific Animations with Sonic Maps , 1989, ICMC.

[54]  Matthew Witten,et al.  Increasing Our Understanding of Biological Models Through Visual and Sonic Representations: a Cortical Case Study , 1992, Int. J. High Perform. Comput. Appl..