The Allobrain: An interactive, stereographic, 3D audio, immersive virtual world

This paper describes the creation of the Allobrain project, an interactive, stereographic, 3D audio, immersive virtual world constructed from fMRI brain data and installed in the Allosphere, one of the largest virtual reality spaces in existence. This paper portrays the role the Allobrain project played as an artwork driving the technological infrastructure of the Allosphere. The construction of the Cosm toolkit software for prototyping the Allobrain and other interactive, stereographic, 3D audio, immersive virtual worlds in the Allosphere is described in detail. Aesthetic considerations of the Allobrain project are discussed in relation to world-making as a means to understand and explore large data sets.

[1]  David G. Malham,et al.  Higher Order Ambisonic Systems for the Spatialisation of Sound , 1999, International Conference on Mathematics and Computing.

[2]  J. Rolland,et al.  Head-worn displays: a review , 2006, Journal of Display Technology.

[3]  Stephen Travis Pope,et al.  Experiencing Audio and Music in a Fully Immersive Environment , 2007, CMMR.

[4]  Karsten Bormann,et al.  Presence and the Utility of Audio Spatialization , 2005, Presence: Teleoperators & Virtual Environments.

[5]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[6]  Matthew Turk,et al.  Creative Collaborative Exploration in Multiple Environments , 2008, AAAI Spring Symposium: Creative Intelligent Systems.

[7]  Tor Langeland,et al.  FAVE – A Framework Architecture for Virtual Environments , 2003 .

[8]  Dylan Menzies,et al.  Ambisonic Synthesis of Complex Sources , 2007 .

[9]  A. J. Berkhout,et al.  A Holographic Approach to Acoustic Control , 1988 .

[10]  Stephen Barrass,et al.  EVALUATION OF A MULTIMODAL SONIFICATION AND VISUALISATION OF DEPTH OF MARKET STOCK DATA , 2002 .

[11]  Tobias Höllerer,et al.  The allosphere: a large-scale immersive surround-view instrument , 2007, EDT '07.

[12]  Helen-Nicole Kostis,et al.  Skin: an interactive hyperstereoscopic electro installation , 2007, Electronic Imaging.

[13]  Taku Komura,et al.  Computing inverse kinematics with linear programming , 2005, VRST '05.

[14]  John M. Chowning,et al.  THE SIMULATION OF MOVING SOUND SOURCES , 1970 .

[15]  John C. Hart,et al.  The CAVE: audio visual experience automatic virtual environment , 1992, CACM.

[16]  Graham Wakefield,et al.  Third-Order Ambisonic Extensions for Max/MSP with Musical Applications , 2006, ICMC.

[17]  Stephen Brewster,et al.  Presenting Dynamic Information on Mobile Computers , 2000 .

[18]  Liliana Ardissono,et al.  Musical interaction design with the CREATE USB interface: teaching HCI with CUIs instead of GUIs , 2008 .

[19]  Jim Kleban,et al.  The multimodal music stand , 2007, NIME '07.

[20]  Michael J. Gerzon.  Periphony: With-Height Sound Reproduction , 1973 .

[21]  SmithJoshua,et al.  Electric Field Sensing For Graphical Interfaces , 1998 .

[22]  Hiroshi Hoshino,et al.  CyberDome: PC Clustered Hemi Spherical Immersive Projection Display , 2003, ICAT.

[23]  Ville Pulkki,et al.  Virtual Sound Source Positioning Using Vector Base Amplitude Panning , 1997 .

[24]  Penny Rheingans,et al.  NIH-NSF visualization research challenges report summary , 2006, IEEE Computer Graphics and Applications.

[25]  Marcos Novak,et al.  Liquid architectures in cyberspace , 1991 .

[26]  Josephine Anstey,et al.  Ygdrasil--a framework for composing shared virtual worlds , 2003, Future Gener. Comput. Syst..

[27]  Stefan Eilemann.  An Analysis of Parallel Rendering Systems , 2007 .

[28]  Georgios Papaioannou,et al.  Media productions for a dome display system , 2006, VRST '06.

[29]  Roger B. Dannenberg,et al.  An Audience-Interactive Multimedia Production on the Brain , 2001 .

[30]  Joshua R. Smith.  Field Mice: Extracting Hand Geometry from Electric Field Measurements , 1996, IBM Syst. J..

[31]  Joseph A. Paradiso,et al.  Musical Applications of Electric Field Sensing , 1997 .

[32]  Sascha Spors,et al.  High-Quality Acoustic Rendering with Wave Field Synthesis , 2002, VMV.

[33]  Jerome Daniel,et al.  Spatial Sound Encoding Including Near Field Effect: Introducing Distance Coding Filters and a Viable, New Ambisonic Format , 2003 .

[34]  Chris R. Johnson,et al.  NHI-NSF Visualization Research Challenges Report , 2005 .

[35]  H. McGurk,et al.  Hearing lips and seeing voices , 1976, Nature.

[36]  A DeFantiThomas,et al.  The CAVE: audio visual experience automatic virtual environment , 1992 .

[37]  David G. Malham,et al.  3-D Sound Spatialization using Ambisonic Techniques , 1995 .

[38]  Roy Kalawsky,et al.  The science of virtual reality and virtual environments - a technical, scientific and engineering reference on virtual environments , 1993 .

[39]  Matti Gröhn,et al.  Application of spatial sound reproduction in virtual environments : experiments in localization, navigation, and orientation , 2006 .

[40]  Johan Ihr,et al.  The Fully Immersive CAVE , 1999 .

[41]  Stephen Travis Pope,et al.  The AlloSphere: Immersive Multimedia for Scientific Discovery and Artistic Exploration , 2009, IEEE MultiMedia.

[42]  Miller Puckette,et al.  Max at Seventeen , 2002, Computer Music Journal.

[43]  Edward J. Wegman,et al.  Immersive Projection Technology for Visual Data Mining , 2002 .