Dynamic shared state maintenance in distributed virtual environments

Advances in computer networks and rendering systems facilitate the creation of distributed collaborative environments in which the distribution of information at remote locations allows efficient communication. Particularly challenging are distributed interactive Virtual Environments (VE) that allow knowledge sharing through 3D information. The purpose of this work is to address the problem of latency in distributed interactive VE and to develop a conceptual model for consistency maintenance in these environments based on the participant interaction model. An area that needs to be explored is the relationship between the dynamic shared state and the interaction with the virtual entities present in the shared scene. Mixed Reality (MR) and VR environments must bring the human participant interaction into the loop through a wide range of electronic motion sensors, and haptic devices. Part of the work presented here defines a novel criterion for categorization of distributed interactive VE and introduces, as well as analyzes, an adaptive synchronization algorithm for consistency maintenance in such environments. As part of the work, a distributed interactive Augmented Reality (AR) testbed and the algorithm implementation details are presented. Currently the testbed is part of several research efforts at the Optical Diagnostics and Applications Laboratory including 3D visualization applications using custom built head-mounted displays (HMDs) with optical motion tracking and a medical training prototype for endotracheal intubation and medical prognostics. An objective method using quaternion calculus is applied for the algorithm assessment. In spite of significant network latency, results show that the dynamic shared state can be maintained consistent at multiple remotely located sites. In further consideration of the latency problems and in the light of the current trends in interactive distributed VE applications, we propose a hybrid distributed system architecture for sensor-based distributed VE that has the potential to improve the system real-time behavior and scalability. (Abstract shortened by UMI.)

[1]  Anne H. Soukhanov,et al.  The american heritage dictionary of the english language , 1992 .

[2]  Hector Garcia-Molina,et al.  Comparing Hybrid Peer-to-Peer Systems , 2001, VLDB.

[3]  Frank Biocca,et al.  A Survey of Position Trackers , 1992, Presence: Teleoperators & Virtual Environments.

[4]  Michael Harrington,et al.  Constellation: a wide-range wireless motion-tracking system for augmented reality and virtual set applications , 1998, SIGGRAPH.

[5]  Richard Campbell.  Managing AFS: The Andrew File System , 1998 .

[6]  Andrew S. Tanenbaum,et al.  Distributed systems: Principles and Paradigms , 2001 .

[7]  Mark A. Livingston,et al.  Superior augmented reality registration by integrating landmark tracking and magnetic tracking , 1996, SIGGRAPH.

[8]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[9]  Luis Serra,et al.  BrickNet: A Software Toolkit for Network-Based Virtual Worlds , 1994, Presence: Teleoperators & Virtual Environments.

[10]  M Fair,et al.  Atlas, a Platform for Distributed Graphics Applications , .

[11]  Jannick P. Rolland,et al.  A Survey of Tracking Technologies for Virtual Environments , 2001 .

[12]  Vinny Cahill,et al.  Taxonomy of distributed event-based programming systems , 2002, Proceedings 22nd International Conference on Distributed Computing Systems Workshops.

[13]  Eric Foxlin,et al.  An inertial head-orientation tracker with automatic drift compensation for use with HMD's , 1994 .

[14]  David L. Mills,et al.  Internet time synchronization: the network time protocol , 1991, IEEE Trans. Commun..

[15]  Brent Callaghan,et al.  NFS Version 3 Protocol Specification , 1995, RFC.

[16]  M. Whitton,et al.  Effect of latency on presence in stressful virtual environments , 2003, IEEE Virtual Reality, 2003. Proceedings..

[17]  P.M. Sharkey,et al.  A local perception filter for distributed virtual environments , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[18]  Paulo Veríssimo,et al.  Distributed Systems for System Architects , 2001, Advances in Distributed Computing and Middleware.

[19]  Mustaque Ahamad,et al.  Slow memory: weakening consistency to enhance concurrency in distributed shared memories , 1990, Proceedings.,10th International Conference on Distributed Computing Systems.

[20]  Tony Ballardie,et al.  Core based trees , 1993 .

[21]  Sandeep K. Singhal,et al.  Exploiting Position History for Efficient Remote Rendering in Networked Virtual Reality , 1995, Presence: Teleoperators & Virtual Environments.

[22]  Tom Davis,et al.  Opengl programming guide: the official guide to learning opengl , 1993 .

[23]  Krishna Kant,et al.  On the Potential of Peer-to-Peer Computing , 2001 .

[24]  Henrik Tramberend,et al.  Avocado: a distributed virtual reality framework , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[25]  D. Schmalstieg,et al.  Distributed applications for collaborative augmented reality , 2002, Proceedings IEEE Virtual Reality 2002.

[26]  Sam Toueg,et al.  A Modular Approach to Fault-Tolerant Broadcasts and Related Problems , 1994 .

[27]  Frank Biocca,et al.  Development of Head-Mounted Projection Displays for Distributed, Collaborative, Augmented Reality Applications , 2005, Presence: Teleoperators & Virtual Environments.

[28]  Michael Peter Hollier,et al.  Multi-modal Perception , 1999 .

[29]  Mingyu Lim,et al.  ATLAS: a scalable network framework for distributed virtual environments , 2002, CVE '02.

[30]  James F. Blinn,et al.  Blue screen matting , 1996, SIGGRAPH.

[31]  Alex Pentland,et al.  Device synchronization using an optimal linear filter , 1992, I3D '92.

[32]  Chris Greenhalgh,et al.  Inside MASSIVE-3: flexible support for data consistency and world structuring , 2000, CVE '00.

[33]  David R. Cheriton,et al.  Amaze: A Multiplayer Computer Game , 1985, IEEE Software.

[34]  Christer Carlsson,et al.  DIVE - A platform for multi-user virtual environments , 1993, Comput. Graph..

[35]  N. Nakamura,et al.  Distributed virtual reality system for cooperative work : Multimedia technology , 1994 .

[36]  Krithi Ramamritham,et al.  Consistency maintenance in peer-to-peer file sharing networks , 2003, Proceedings the Third IEEE Workshop on Internet Applications. WIAPP 2003.

[37]  Maxine D. Brown,et al.  The ImmersaDesk and Infinity Wall projection-based virtual reality displays , 1997, COMG.

[38]  Chandrajit L. Bajaj,et al.  Distributed and collaborative visualization , 1994, Computer.

[39]  Steve Pettifer,et al.  DEVA3: architecture for a large-scale distributed virtual reality system , 2000, VRST '00.

[40]  Scott Stevens Fisher,et al.  Viewpoint Dependent Imaging: An Interactive Stereoscopic Display , 1983, Optics & Photonics.

[41]  David Gelernter,et al.  Generative communication in Linda , 1985, TOPL.

[42]  David S. Rosenblum,et al.  Issues in supporting event-based architectural styles , 1998, ISAW '98.

[43]  David R. Pratt,et al.  A software architecture for the construction and management of real-time virtual worlds. , 1993 .

[44]  S. Ellis.  Pictorial communication in virtual and real environments , 1991 .

[45]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[46]  Frank Biocca,et al.  Evolution of the Mobile Infosphere : Iterative design of a high infomation-bandwidth , mobile augmented reality interface , 2022 .

[47]  Craig Partridge,et al.  Flow synchronization protocol , 1994, TNET.

[48]  Ng Hern,et al.  BrickNet: sharing object behaviors on the Net , 1995, Proceedings Virtual Reality Annual International Symposium '95.

[49]  Henry Fuchs,et al.  A real-time optical 3D tracker for head-mounted display systems , 1990, I3D '90.

[50]  John R. Aschenbrenner,et al.  Open Systems Interconnection , 1986, IBM Syst. J..

[51]  Nicholas Carriero,et al.  A computational model of everything , 2001, CACM.

[52]  Edward J. Wegman.  Affordable environments for 3D collaborative data visualization , 2000, Comput. Sci. Eng..

[53]  Ken Brodlie,et al.  Distributed and Collaborative Visualization , 2004, Comput. Graph. Forum.

[54]  J. Mcneff.  The global positioning system , 2002 .

[55]  Suya You,et al.  Augmented Reality Tracking in Natural Environments , 1998 .

[56]  Richard C. Waters,et al.  Locales: supporting large multiuser virtual environments , 1996, IEEE Computer Graphics and Applications.

[57]  Michael Zyda,et al.  Three-tiered interest management for large-scale virtual environments , 1998, VRST '98.

[58]  Michael Zyda,et al.  NPSNET:A Network Software Architecture for LargeScale Virtual Environments , 1994, Presence: Teleoperators & Virtual Environments.

[59]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[60]  Larry S. Davis,et al.  Enabling a Continuum of Virtual Environment Experiences , 2003, IEEE Computer Graphics and Applications.

[61]  Andrew Lippman,et al.  Movie-maps: An application of the optical videodisc to computer graphics , 1980, SIGGRAPH '80.

[62]  Jannick P. Rolland,et al.  Optical versus Video See-Through Head-Mounted Displays , 2001 .

[63]  Frank J. Ferrin.  Survey of helmet tracking technologies , 1991, Medical Imaging.

[64]  Jack B. Kuipers,et al.  Quaternions and Rotation Sequences: A Primer with Applications to Orbits, Aerospace and Virtual Reality , 2002 .

[65]  Sandeep K. Singhal,et al.  Log-based receiver-reliable multicast for distributed interactive simulation , 1995, SIGCOMM '95.

[66]  Michael Zyda,et al.  NPSNET: constructing a 3D virtual world , 1992, I3D '92.

[67]  Jannick P. Rolland,et al.  The ARC Display: an augmented reality visualization center , 2002, The First IEEE International Workshop Agumented Reality Toolkit,.

[68]  Franco Zambonelli,et al.  Mobile-Agent Coordination Models for Internet Applications , 2000, Computer.

[69]  David J. Roberts,et al.  Maximising concurrency and scalability in a consistent, causal, distributed virtual reality system, whilst minimising the effect of network delays , 1997, Proceedings of IEEE 6th Workshop on Enabling Technologies: Infrastructure for Collaborative Enterprises.

[70]  Dieter Schmalstieg,et al.  Distributed Open Inventor: a practical approach to distributed 3D graphics , 1999, VRST '99.

[71]  Jason Daly,et al.  Development of a training tool for endotracheal intubation: distributed augmented reality. , 2003, Studies in health technology and informatics.

[72]  Bo Leuf,et al.  Peer to Peer: Collaboration and Sharing over the Internet , 2002 .

[73]  Michael Zyda,et al.  NPSNET-V: A New Beginning for Dynamically Extensible Virtual Environments , 2000, IEEE Computer Graphics and Applications.

[74]  D. C. Miller,et al.  SIMNET: the advent of simulator networking , 1995, Proc. IEEE.

[75]  Jack A. Thorpe,et al.  The New Technology of Large Scale Simulator Networking: Implications for Mastering the Art of Warfighting , 1987 .

[76]  Frank Biocca,et al.  Teleportal Augmented Reality System: Integrating Virtual Objects, Remote Collaborators, and Physical Reality for Distributed Networked Manufacturing , 2004 .

[77]  Felix G. Hamza-Lup,et al.  Physically-based Deformation of High-Resolution 3D Lung Models for Augmented Reality based Medical Visualization , 2004 .

[78]  Michael Zyda,et al.  Networked virtual environments - desgin and implementation , 1999 .

[79]  Mineo Takai,et al.  Parssec: A Parallel Simulation Environment for Complex Systems , 1998, Computer.

[80]  Sandeep K. Singhal,et al.  Using projection aggregations to support scalability in distributed simulation , 1996, Proceedings of 16th International Conference on Distributed Computing Systems.

[81]  Felix G. Hamza-Lup,et al.  A Less Intrusive System Monitoring Scheme for Distributed Virtual Environments , 2004 .

[82]  Qin Xiao,et al.  Delayed Consistency Model for Distributed Interactive Systems with Real-Time Continuous Media , 2002 .

[83]  Marvin Theimer,et al.  The Case for Non-transparent Replication: Examples from Bayou , 1998, IEEE Data Eng. Bull..

[84]  Colin Swindells,et al.  System lag tests for augmented and virtual environments , 2000, UIST '00.

[85]  Brett D. Fleisch,et al.  A Case for Virtual Distributed Objects , 1998, Parallel Distributed Comput. Pract..

[86]  Stephen R. Ellis,et al.  Localization of Virtual Objects in the Near Visual Field , 1998, Hum. Factors.

[87]  Glenn A. Martin,et al.  VESS: coordinating graphics, audio, and user interaction in virtual reality applications , 2002, Proceedings IEEE Virtual Reality 2002.

[88]  Krishna Kant,et al.  An analytic model for peer to peer file sharing networks , 2003, IEEE International Conference on Communications, 2003. ICC '03..

[89]  Larry S. Davis,et al.  A method for designing marker-based tracking probes , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[90]  Carolina Cruz-Neira,et al.  Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE , 2023 .

[91]  Emmanuel Frécon,et al.  DIVE: a scaleable network architecture for distributed virtual environments , 1998, Distributed Syst. Eng..

[92]  Nathaniel I. Durlach,et al.  Telepresence, time delay and adaptation , 1991 .

[93]  Ladislau Bölöni,et al.  The Bond Agent System and Applications , 2000, ASA/MA.

[94]  Steven K. Feiner,et al.  A distributed 3D graphics library , 1998, SIGGRAPH.

[95]  Matthew Lombard,et al.  At the Heart of It All: The Concept of Presence , 2006 .

[96]  Alexei A. Goon,et al.  A SURVEY OF TRACKING TECHNOLOGY FOR VIRTUAL ENVIRONMENTS , 1999 .