Immersive full-surround multi-user system design

This paper describes our research in full-surround, multimodal, multi-user, immersive instrument design in a large VR instrument. The three-story instrument, designed for large-scale, multimodal representation of complex and potentially high-dimensional information, specifically focuses on multi-user participation by facilitating interdisciplinary teams of co-located researchers in exploring complex information through interactive visual and aural displays in a full-surround, immersive environment. We recently achieved several milestones in the instrument's design that improves multi-user participation when exploring complex data representations and scientific simulations. These milestones include affordances for ''ensemble-style'' interaction allowing groups of participants to see, hear, and explore data as a team using our multi-user tracking and interaction systems; separate visual display modes for rectangular legacy content and for seamless surround-view stereoscopic projection using 4 high-resolution, high-lumen projectors with hardware warping and blending integrated with 22 small-footprint projectors placed above and below the instrument's walkway; and a 3D spatial audio system enabling a variety of sound spatialization techniques. These facilities can be accessed and controlled by a multimodal framework for authoring applications integrating visual, audio, and interactive elements. We report on the achieved instrument design.

[1]  Falko Kuester,et al.  The StarCAVE, a third-generation CAVE and virtual reality OptIPortal , 2009, Future Gener. Comput. Syst..

[2]  Lance Putnam,et al.  Dynamic Interactivity Inside the AlloSphere , 2010, NIME.

[3]  Matthew Wright,et al.  Sound Element Spatializer , 2011, ICMC.

[4]  John C. Hart,et al.  The CAVE: audio visual experience automatic virtual environment , 1992, CACM.

[5]  Kallirroi Georgila,et al.  Practical Evaluation of Human and Synthesized Speech for Virtual Human Dialogue Systems , 2012, LREC.

[6]  Brian Davis,et al.  Geowall: Investigations into Low-Cost Stereo Display Technologies , 2003 .

[7]  Graham Wakefield,et al.  Augmenting Computer Music with Just-In-Time Compilation , 2009, ICMC.

[8]  JoAnn Kuchera-Morin Performing in Quantum Space: A Creative Approach to N-Dimensional Computing , 2011, Leonardo.

[9]  Stephen Travis Pope,et al.  The AlloSphere: Immersive Multimedia for Scientific Discovery and Artistic Exploration , 2009, IEEE MultiMedia.

[10]  Graham Wakefield,et al.  Immersed in Unfolding Complex Systems , 2010, Beautiful Visualization.

[11]  Kyriakos G. Vamvoudakis,et al.  A Mission-Centric Visualization Tool for Cybersecurity Situation Awareness , 2012 .

[12]  Carlo A. Marzi Two brains, one clock , 2004, Trends in Cognitive Sciences.

[13]  Matthew Wright,et al.  Open SoundControl: A New Protocol for Communicating with Sound Synthesizers , 1997, ICMC.

[14]  Graham Wakefield,et al.  COSM: A toolkit for composing immersive audio-visual worlds of agency and autonomy , 2011, ICMC.

[15]  H. Bülthoff,et al.  Merging the senses into a robust percept , 2004, Trends in Cognitive Sciences.

[16]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[17]  Jérémie Gerhardt,et al.  Interactive 3D Stereoscopic Dome with Automatic Calibration , 2010, Eurographics.

[18]  Andrew Zisserman,et al.  Multiple view geometry in computer visiond , 2001 .

[19]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[20]  Stephen Travis Pope,et al.  Experiencing Audio and Music in a Fully Immersive Environment , 2007, CMMR.

[21]  Mike Bailey,et al.  Realtime Dome Imaging and Interaction: Towards Immersive Design Environments , 2006 .

[22]  Andreas Simon,et al.  Omnistereo for panoramic virtual environment display systems , 2004, IEEE Virtual Reality 2004.

[23]  Georgios Papaioannou,et al.  Designing a Real-Time playback system for a Dome Theater , 2006 .

[24]  David Zicarelli How I Learned to Love a Program That Does Nothing , 2002, Computer Music Journal.

[25]  Charles Roberts Control: Software for End-User Interface Programming and Interactive Performance , 2011, ICMC.

[26]  Stephen Pope,et al.  Acoustics of a large immersive environment, The Allosphere , 2008 .

[27]  Tobias Höllerer,et al.  Augmented textual data viewing in 3D visualizations using tablets , 2012, 2012 IEEE Symposium on 3D User Interfaces (3DUI).

[28]  Tobias Höllerer,et al.  The allosphere: a large-scale immersive surround-view instrument , 2007, EDT '07.

[29]  Casey Reas,et al.  Processing: programming for the media arts , 2006, AI & SOCIETY.

[30]  Richard May,et al.  A Survey of Large High-Resolution Display Technologies, Techniques, and Applications , 2006, IEEE Virtual Reality Conference (VR 2006).

[31]  C. Cruz-Neira,et al.  A 'room' with a$ 'view' , 1993, IEEE Spectrum.

[32]  Graham WAKEFIELD,et al.  LuaAV : Extensibility and Heterogeneity for Audiovisual Computing , 2010 .

[33]  B. Hamann,et al.  Interactive terrain visualization enables virtual field work during rapid scientific response to the 2010 Haiti earthquake , 2012 .

[34]  Stephen Travis Pope,et al.  Immersive audio and Music in the AlloSphere , 2007, ICMC.

[35]  U. Eco,et al.  The Future of the Book , 1996 .

[36]  Vid Petrovic,et al.  The future of the CAVE , 2011 .

[37]  Ed Lantz,et al.  A survey of large-scale immersive displays , 2007, EDT '07.

[38]  Thomas A. DeFanti,et al.  AGAVE : Access Grid Augmented Virtual Environment , 2001 .

[39]  Pere Brunet,et al.  MiniVR: a portable virtual reality system , 2004, Comput. Graph..

[40]  Aditi Majumder,et al.  Auto-registration techniques for immersive multi-projector displays , 2012 .

[41]  R. Comerford Technology 1993 - Software , 1993, IEEE Spectrum.