3D Virtual worlds and the metaverse: Current status and future possibilities

Moving from a set of independent virtual worlds to an integrated network of 3D virtual worlds or Metaverse rests on progress in four areas: immersive realism, ubiquity of access and identity, interoperability, and scalability. For each area, the current status and needed developments in order to achieve a functional Metaverse are described. Factors that support the formation of a viable Metaverse, such as institutional and popular interest and ongoing improvements in hardware performance, and factors that constrain the achievement of this goal, including limits in computational methods and unrealized collaboration among virtual world stakeholders and developers, are also considered.

[1]  A. Bentur,et al.  Terminology and definitions , 2003 .

[2]  Derek Bradley,et al.  High resolution passive facial performance capture , 2010, ACM Trans. Graph..

[3]  Takeo Igarashi,et al.  Interaction: interfaces, algorithms, and applications , 2009, SIGGRAPH '09.

[4]  Richard O. Duda,et al.  A structural model for binaural sound synthesis , 1998, IEEE Trans. Speech Audio Process..

[5]  Musawir A. Shah,et al.  Image-space approach to real-time realistic rendering , 2007 .

[6]  Johannes Gehrke,et al.  Scalability for Virtual Worlds , 2009, 2009 IEEE 25th International Conference on Data Engineering.

[7]  Matthias Harders,et al.  Multimodal Augmented Reality in Medicine , 2007, HCI.

[8]  Paul Debevec,et al.  The Digital Emily project: photoreal facial modeling and animation , 2009, SIGGRAPH '09.

[9]  Zoran Popović,et al.  Motion fields for interactive character locomotion , 2010, SIGGRAPH 2010.

[10]  Dinesh Manocha,et al.  Sounding liquids: Automatic sound synthesis from fluid simulation , 2010, TOGS.

[11]  Frank Biocca,et al.  Visual Touch in Virtual Environments: An Exploratory Study of Presence, Multimodal Interfaces, and Cross-Modal Sensory Illusions , 2001, Presence: Teleoperators & Virtual Environments.

[12]  Remi Arnaud,et al.  COLLADA: Sailing the Gulf of 3D Digital Content Creation , 2006 .

[13]  G.E. Moore,et al.  Cramming More Components Onto Integrated Circuits , 1998, Proceedings of the IEEE.

[14]  Hongxia Wang,et al.  Realistic Real-Time Rendering of 3D Terrain Scenes Based on OpenGL , 2009, 2009 First International Conference on Information Science and Engineering.

[15]  Martin R. Gibbs,et al.  You can be too rich: mediated communication in a virtual world , 2009, OZCHI.

[16]  Yael Edan,et al.  Vision-based hand-gesture applications , 2011, Commun. ACM.

[17]  R. Kurzweil.  The Law of Accelerating Returns , 2004 .

[18]  Weifeng Sun,et al.  Generalized wavelet product integral for rendering dynamic glossy objects , 2006, SIGGRAPH '06.

[19]  Alexander Zelinsky,et al.  Real-time stereo tracking for head pose and gaze estimation , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[20]  Doug L. James,et al.  Harmonic fluids , 2009, ACM Trans. Graph..

[21]  Daijin Kim,et al.  Real-time facial expression recognition using STAAM and layered GDA classifier , 2009, Image Vis. Comput..

[22]  Doug L. James,et al.  Harmonic shells: a practical nonlinear sound model for near-rigid thin shells , 2009, SIGGRAPH 2009.

[23]  George N. Phillips,et al.  Modular approach of multimodal integration in a virtual environment , 2002, Proceedings. Fourth IEEE International Conference on Multimodal Interfaces.

[24]  Aimin Hao,et al.  Real-Time Realistic Rendering of Tissue Surface with Mucous Layer , 2009, 2009 Second International Workshop on Computer Science and Engineering.

[25]  Jim Waldo,et al.  Scaling in games and virtual worlds , 2008, CACM.

[26]  Jörn Hurtienne,et al.  Multimodal interaction: A suitable strategy for including older users? , 2010, Interact. Comput..

[27]  Zoran Popovic,et al.  Motion fields for interactive character locomotion , 2010, CACM.

[28]  Anatole Lécuyer,et al.  Simulating Haptic Feedback Using Vision: A Survey of Research and Applications of Pseudo-Haptic Feedback , 2009, PRESENCE: Teleoperators and Virtual Environments.

[29]  Weifeng Sun,et al.  Generalized wavelet product integral for rendering dynamic glossy objects , 2006, SIGGRAPH 2006.

[30]  Maximo Cobos,et al.  A Sparsity-Based Approach to 3D Binaural Sound Synthesis Using Time-Frequency Array Processing , 2010, EURASIP J. Adv. Signal Process..

[31]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[32]  Dave Morris,et al.  Machinima: Making Animated Movies in 3D Virtual Environments , 2005 .

[33]  David P. Reed.  Designing croquet's TeaTime: a real-time, temporal environment for active object cooperation , 2005, OOPSLA '05.

[34]  Chen Shen,et al.  Synthesizing sounds from rigid-body simulations , 2002, SCA '02.

[35]  Xu Yang,et al.  Real-time rendering of realistic rain , 2006, SIGGRAPH '06.

[36]  共立出版株式会社.  コンピュータ・サイエンス : ACM computing surveys , 1978 .

[37]  Sumanta N. Pattanaik,et al.  Rendering Grass in Real Time with Dynamic Lighting , 2009, IEEE Computer Graphics and Applications.

[38]  Marc Erich Latoschik.  A user interface framework for multimodal VR interactions , 2005, ICMI '05.

[39]  N DionisioJohn David,et al.  3D Virtual worlds and the metaverse , 2013 .

[40]  Gwendal Simon,et al.  Toward a peer-to-peer shared virtual reality , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[41]  Myron W. Krueger.  An Easy Entry Artificial Reality , 1993 .

[42]  Sophie Jörg,et al.  Investigating the role of body shape on the perception of emotion , 2009, TAP.

[43]  Nadia Magnenat-Thalmann,et al.  Real-time rendering of woven clothes , 2003, VRST '03.

[44]  Caroline Jay,et al.  Modeling the effects of delayed haptic and visual feedback in a collaborative virtual environment , 2007, TCHI.

[45]  Choon Seong Leem,et al.  A taxonomy of ubiquitous computing applications , 2007, Wirel. Pers. Commun..

[46]  Anselmo Lastra,et al.  A shading language on graphics hardware: the pixelflow shading system , 1998, SIGGRAPH.

[47]  Abu Saleh Md. Mahfujur Rahman,et al.  Bridging the Gap between Virtual and Real World by Bringing an Interpersonal Haptic Communication System in Second Life , 2010, 2010 IEEE International Symposium on Multimedia.

[48]  Hakan Gurocak,et al.  Haptic Glove with MR Brakes for Distributed Finger Force Feedback , 2009, PRESENCE: Teleoperators and Virtual Environments.

[49]  S. Turkle.  Life on the Screen: Identity in the Age of the Internet , 1997 .

[50]  Michael D. McCool,et al.  Real-time shading , 2002, SIGGRAPH '04.

[51]  F. R. Farmer,et al.  The lessons of Lucasfilm's habitat , 1991 .

[52]  Anne-Marie Kermarrec,et al.  Solipsis: A Decentralized Architecture for Virtual Environments , 2008 .

[53]  Jaron Lanier,et al.  Reality built for two: a virtual reality tool , 1990, I3D '90.

[54]  John Short,et al.  The social psychology of telecommunications , 1976 .

[55]  Eric S. Raymond,et al.  The cathedral and the bazaar - musings on Linux and Open Source by an accidental revolutionary , 2001 .

[56]  Diego Gutierrez,et al.  Exploiting perception in high-fidelity virtual environments (Additional presentations from the 24th course are available on the citation page) , 2006, SIGGRAPH Courses.

[57]  Sergio Martinoia,et al.  Coding and decoding of information in a bi-directional neural interface , 2005, Neurocomputing.

[58]  J. Blauert.  Spatial Hearing: The Psychophysics of Human Sound Localization , 1983 .

[59]  Jaron Lanier,et al.  An Insider's View of the Future of Virtual Reality , 1992 .

[60]  Andrea F. Abate,et al.  A haptic-based approach to virtual training for aerospace industry , 2009, J. Vis. Lang. Comput..

[61]  Nelson L. Max,et al.  Interactive multiple anisotropic scattering in clouds , 2008, I3D '08.

[62]  Dinesh Manocha,et al.  RESound: interactive sound rendering for dynamic virtual environments , 2009, ACM Multimedia.

[63]  Peter Ludlow,et al.  The Second Life Herald: The Virtual Tabloid that Witnessed the Dawn of the Metaverse , 2007 .

[64]  Lynne Baillie,et al.  Rolling, rotating and imagining in a virtual mobile world , 2005, Mobile HCI.

[65]  Doug L. James,et al.  Harmonic shells: a practical nonlinear sound model for near-rigid thin shells , 2009, ACM Trans. Graph..

[66]  Mic Bowman,et al.  Scaling virtual worlds: Simulation requirements and challenges , 2010, Proceedings of the 2010 Winter Simulation Conference.

[67]  Artur Lugmayr,et al.  Proceedings of the 13th International MindTrek Conference: Everyday Life in the Ubiquitous Era , 2009 .

[68]  S. Turkle,et al.  Evocative objects : things we think with , 2007 .

[69]  Mark Grimshaw,et al.  Bridging the uncanny: an impossible traverse? , 2009, MindTrek '09.

[70]  George Drettakis,et al.  Fast modal sounds with scalable frequency-domain synthesis , 2008, ACM Trans. Graph..

[71]  Christine L. Lisetti,et al.  Toward multimodal fusion of affective cues , 2006, HCM '06.

[72]  Doug L. James,et al.  Rigid-body fracture sound with precomputed soundbanks , 2010, ACM Trans. Graph..

[73]  G. Recanzone.  Auditory influences on visual temporal rate perception. , 2003, Journal of neurophysiology.

[74]  Jim Everett,et al.  Coming of Age in Second Life: An Anthropologist Explores the Virtually Human , 2010 .

[75]  Mohamad Sawan,et al.  A low-power integrated neural interface with digital spike detection and extraction , 2010 .

[76]  Richard L. Gilbert,et al.  The Distributed Self: Virtual Worlds and the Future of Human Identity , 2013 .

[77]  Martin Breidt,et al.  Face reality: investigating the Uncanny Valley for virtual faces , 2010, SIGGRAPH ASIA.

[78]  Christophe d'Alessandro,et al.  3D Audiovisual Rendering and Real-Time Interactive Control of Expressivity in a Talking Head , 2007, IVA.

[79]  Karl F. MacDorman,et al.  Human emotion and the uncanny valley: A GLM, MDS, and Isomap analysis of robot video ratings , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[80]  Michael Koch,et al.  Ubiquitous Computing , 2001, CSCW-Kompendium.

[81]  Roy T. Fielding,et al.  Uniform Resource Identifier (URI): Generic Syntax , 2005, RFC.

[82]  Junchul Chun,et al.  Real-Time Face Pose Tracking and Facial Expression Synthesizing for the Animation of 3D Avatar , 2007, Edutainment.

[83]  Eduardo Zalama Casanova,et al.  A realistic, virtual head for human-computer interaction , 2010, Interact. Comput..

[84]  Baining Guo,et al.  Realistic, real‐time rendering of ocean waves , 2006, Comput. Animat. Virtual Worlds.

[85]  Franck Multon,et al.  Influence of the Graphical Levels of Detail of a Virtual Thrower on the Perception of the Movement , 2010, PRESENCE: Teleoperators and Virtual Environments.

[86]  Mark Grimshaw,et al.  Facial expression of emotion and perception of the Uncanny Valley in virtual characters , 2011, Comput. Hum. Behav..

[87]  Eva-Lotta Sallnäs,et al.  Haptic Feedback Increases Perceived Social Presence , 2010, EuroHaptics.

[88]  Mitsuru Ishizuka,et al.  iFeel_IM: innovative real-time communication system with rich emotional and haptic channels , 2010, CHI Extended Abstracts.

[89]  Abhijeet Ghosh.  Realistic materials and illumination environments , 2007 .

[90]  Catrin Misselhorn,et al.  Empathy with Inanimate Objects and the Uncanny Valley , 2009, Minds and Machines.

[91]  Doug L. James,et al.  Rigid-body fracture sound with precomputed soundbanks , 2010, ACM Trans. Graph..

[92]  Musawir A. Shah,et al.  Image-Space Subsurface Scattering for Interactive Rendering of Deformable Translucent Objects , 2009, IEEE Computer Graphics and Applications.

[93]  Baining Guo,et al.  Realistic, real-time rendering of ocean waves: Research Articles , 2006 .

[94]  M. Weiser,et al.  Hot topics-ubiquitous computing , 1993 .

[95]  Christoph Trattner,et al.  Socializing virtual worlds with Facebook: a prototypical implementation of an expansion pack to communicate between Facebook and OpenSimulator based virtual worlds , 2010, MindTrek.

[96]  Ning Xu,et al.  A Novel Voice Morphing System Using Bi-GMM for High Quality Transformation , 2008, 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing.

[97]  Ina Fourie,et al.  The Second Life Herald: The Virtual Tabloid that Witnessed the Dawn of the Metaverse , 2009 .

[98]  Francis K. H. Quek,et al.  The Effects of Finger-Walking in Place (FWIP) for Spatial Knowledge Acquisition in Virtual Environments , 2010, Smart Graphics.

[99]  Ben Liang,et al.  Proceedings of the 2008 Ambi-Sys workshop on Haptic user interfaces in ambient media systems , 2008, Ambi-sys 2008.

[100]  Rob Aspin,et al.  A GPU based, projective multi-texturing approach to reconstructing the 3D human form for application in tele-presence , 2011, CSCW '11.

[101]  John C. Hart,et al.  The CAVE: audio visual experience automatic virtual environment , 1992, CACM.

[102]  Wijnand A. IJsselsteijn,et al.  Mediated social touch: a review of current research and future directions , 2006, Virtual Reality.

[103]  Jaron Lanier,et al.  Virtual reality: the promise of the future , 1992 .

[104]  Jun'ichiro Seyama,et al.  Probing the Uncanny Valley with the Eye Size Aftereffect , 2009, PRESENCE: Teleoperators and Virtual Environments.

[105]  Sriram Subramanian,et al.  Haptic feedback in remote pointing , 2009, CHI Extended Abstracts.

[106]  Tom Boellstorff.  Coming of Age in Second Life: An Anthropologist Explores the Virtually Human , 2008 .

[107]  Karon E. MacLean,et al.  Seeing, hearing, and touching: putting it all together , 2004, SIGGRAPH '04.

[108]  Rod McCall,et al.  Infinite Reality: Avatars, Eternal Life, New Worlds, and the Dawn of the Virtual Revolution , 2011, PRESENCE: Teleoperators and Virtual Environments.

[109]  Domenico Prattichizzo,et al.  Bringing haptics to Second Life , 2008, HAS '08.

[110]  W. Heidrich,et al.  High resolution passive facial performance capture , 2010, ACM Trans. Graph..

[111]  John Charles.  Neural Interfaces Link the Mind and the Machine [Industry Trends] , 1999, Computer.

[112]  John David N. Dionisio,et al.  Texture map based sound synthesis in rigid-body simulations , 2008, International Conference on Computer Graphics and Interactive Techniques.

[113]  James G. Phillips,et al.  Human visual perception of region warping distortions , 2006, ACSC.

[114]  Sebastian Thrun,et al.  Video-based reconstruction of animatable human characters , 2010, ACM Trans. Graph..

[115]  William R. Sherman,et al.  RIST: Radiological Immersive Survey Training for two simultaneous users , 2010, Comput. Graph..

[116]  Shinjiro Kawato,et al.  Real-time detection of nodding and head-shaking by directly detecting and tracking the "between-eyes" , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[117]  Katherine J. Kuchenbecker.  Haptography: capturing the feel of real objects to enable authentic haptic rendering (invited paper) , 2008, HAS '08.

[118]  Gordon Ellis,et al.  Grand challenges for engineering , 2009, IEEE Engineering Management Review.

[119]  Carl Gutwin,et al.  Haptic and Tactile Feedback in Directed Movements , 2005 .

[120]  Qinping Zhao,et al.  10 scientific problems in virtual reality , 2011, Commun. ACM.

[121]  Doug L. James,et al.  Harmonic fluids , 2009, SIGGRAPH 2009.

[122]  Diego Gutierrez,et al.  A practical appearance model for dynamic facial color , 2010, ACM Trans. Graph..

[123]  Jochen Triesch,et al.  Robotic Gesture Recognition by Cue Combination , 1998, GI Jahrestagung.

[124]  Jun'ichiro Seyama,et al.  The Uncanny Valley: Effect of Realism on the Impression of Artificial Human Faces , 2007, PRESENCE: Teleoperators and Virtual Environments.

[125]  Roy T. Fielding,et al.  Uniform Resource Identifiers (URI): Generic Syntax , 1998, RFC.

[126]  Pradeep Dubey,et al.  Second Life and the New Generation of Virtual Worlds , 2008, Computer.

[127]  Giuseppina C. Gini,et al.  A Multi-Modal Haptic Interface for Virtual Reality and Robotics , 2008, J. Intell. Robotic Syst..

[128]  A. Vickers,et al.  Terminology and Definitions , 2003 .

[129]  Richard L. Gilbert.  The P.R.O.S.E. (Psychological Research on Synthetic Environments) Project: Conducting In-World Psychological Research on 3D Virtual Worlds , 1970 .

[130]  Masahiko Inami,et al.  ImpAct: Immersive haptic stylus to enable direct touch and manipulation for surface computing , 2010, CIE.

[131]  Jochen Triesch,et al.  Vision in natural and virtual environments , 2002, ETRA.

[132]  Dzmitry Tsetserukou,et al.  HaptiHug: A Novel Haptic Display for Communication of Hug over a Distance , 2010, EuroHaptics.

[133]  Tadashi Sakata,et al.  Vowel Synthesis Based on the Spectral Morphing and Its Application to Speaker Conversion , 2006, First International Conference on Innovative Computing, Information and Control - Volume I (ICICIC'06).

[134]  Oskar Elek,et al.  Real-time spectral scattering in large-scale natural participating media , 2010, SCCG.

[135]  Randi J. Rost.  OpenGL shading language , 2004 .

[136]  Akash I. Mecwan,et al.  Voice conversion algorithm , 2009, ICAC3 '09.

[137]  Wan Bok Lee,et al.  Image blending for virtual environment construction based on TIP model , 2005 .

[138]  Jaron Lanier,et al.  You Are Not a Gadget: A Manifesto , 2010 .

[139]  Richard L. Gilbert,et al.  Sexuality in the 3D Internet and its relationship to real-life sexuality , 2011 .

[140]  Ken Perlin,et al.  Realistic human body movement for emotional expressiveness , 2009, SIGGRAPH '09.

[141]  Sherry Turkle,et al.  Evocative objects : things we think with , 2007 .

[142]  Nicolai Marquardt,et al.  Proxemic interactions: the new ubicomp? , 2011, INTR.

[143]  Dinesh K. Pai,et al.  FoleyAutomatic: physically-based sound effects for interactive simulation and animation , 2001, SIGGRAPH.

[144]  Takeshi Naemura,et al.  Internet communication using real-time facial expression analysis and synthesis , 2004, IEEE MultiMedia.

[145]  Gerhard Schwabe,et al.  Audio vs. Chat: The Effects of Group Size on Media Choice , 2007, 2007 40th Annual Hawaii International Conference on System Sciences (HICSS'07).

[146]  Richard L. Gilbert,et al.  Multiple Personality Order: Physical and Personality Characteristics of the Self, Primary Avatar and Alt , 2011 .