Performance Analysis of Distributed Virtual Environments

A distributed virtual environment (DVE) is a shared virtual environment where multiple users at their workstations interact with each other. Some of these systems may support a large number of users, e.g., massive multi-player online games, and these users may be geographically distributed. An important performance measure in a DVE system is the delay for an update of a user's state (e.g., his position in the virtual environment) to arrive at the workstations of those users who are affected by the update. This update delay often has a stringent requirement (e.g., less than 100 ms) in order to ensure interactivity among users. In designing a DVE system, an important issue is how well the system scales as the number of users increases. In terms of scalability, a promising system architecture is a two-level hierarchical architecture. At the lower level, multiple service facilities (or basic systems) are deployed; each basic system interacts with its assigned users. At the higher level, the various basic systems ensure that their copies of the virtual environment are as consistent as possible. Although this architecture is believed to have good properties with respect to scalability, not much is known about its performance characteristics. This thesis is concerned with the performance characteristics of the two-level hierarchical architecture. We first investigate the issue of scalability. We obtain analytic results on the workload experienced by the various basic systems as a function of the number of users. Our results provide valuable insights into the scalability of the architecture. We also propose a novel technique to achieve weak consistency among copies of the virtual environment at the various basic systems. Simulation results on the consistency/scalability tradeoff are presented. We next study the update delay in the two-level hierarchical architecture. The update delay has two main components, namely the delay at the basic system (or server delay) and the network delay. For the server delay, we use a network of queues model where each basic system may have one or more processors. We develop an approximation method to obtain results for the distribution of server delay. Comparisons with simulation show that our approximation method yields accurate results. We also measure the time to process an update on an existing online game server. Our approximate results are then used to characterize the 95th-percentile of the server delay, using the measurement data as input. As to the network delay, we develop a general network model and obtain analytic results for the network delay distribution. Numerical examples are presented to show the conditions under which geographical distribution of basic systems will lead to an improvement in the network delay. We also develop an efficient heuristic algorithm that can be used to determine the best locations for the basic systems in a network.

[1]  O. Kariv,et al.  An Algorithmic Approach to Network Location Problems. II: The p-Medians , 1979 .

[2]  Li Xiao,et al.  Dynamic Layer Management in Superpeer Architectures , 2005, IEEE Trans. Parallel Distributed Syst..

[3]  Mark Claypool,et al.  The effect of latency on user performance in Warcraft III , 2003, NetGames '03.

[4]  B. Neuman Scale in Distributed Systems , 1994 .

[5]  Hector Garcia-Molina,et al.  Designing a super-peer network , 2003, Proceedings 19th International Conference on Data Engineering (Cat. No.03CH37405).

[6]  Mark Claypool,et al.  The effects of latency on online madden NFL football , 2004, NOSSDAV '04.

[7]  Yves Lepage,et al.  MBONE, multicasting tomorrow's Internet , 1996 .

[8]  Eric Cronin,et al.  A Distributed Multiplayer Game Server System , 2001 .

[9]  Mark Claypool,et al.  The effects of loss and latency on user performance in unreal tournament 2003® , 2004, NetGames '04.

[10]  P. Krishnan,et al.  The cache location problem , 2000, TNET.

[11]  John C. S. Lui,et al.  An Efficient Partitioning Algorithm for Distributed Virtual Environment Systems , 2002, IEEE Trans. Parallel Distributed Syst..

[12]  Michael S. Borella,et al.  Source models of network game traffic , 2000, Comput. Commun..

[13]  Thomas A. Funkhouser,et al.  RING: a client-server system for multi-user virtual environments , 1995, I3D '95.

[14]  Michael Zyda,et al.  NPSNET-V: A New Beginning for Dynamically Extensible Virtual Environments , 2000, IEEE Computer Graphics and Applications.

[15]  D. C. Miller,et al.  SIMNET: the advent of simulator networking , 1995, Proc. IEEE.

[16]  Bo Li,et al.  On the optimal placement of web proxies in the Internet , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[17]  Thomas A. Funkhouser,et al.  Network topologies for scalable multi-user virtual environments , 1996, Proceedings of the IEEE 1996 Virtual Reality Annual International Symposium.

[18]  Angelos Bilas,et al.  Behavior and Performance of Interactive Multi-Player Game Servers , 2001, 2001 IEEE International Symposium on Performance Analysis of Systems and Software. ISPASS..

[19]  Gurminder Singh,et al.  NetEffect: a network architecture for large-scale multi-user virtual worlds , 1997, VRST '97.

[20]  Wu-chi Feng,et al.  Provisioning on-line games: a traffic analysis of a busy counter-strike server , 2002, CCRV.

[21]  Luis Serra,et al.  BrickNet: A Software Toolkit for Network-Based Virtual Worlds , 1994, Presence: Teleoperators & Virtual Environments.

[22]  Rory Stuart,et al.  Design of Virtual Environments , 2001 .

[23]  Ellen W. Zegura,et al.  A quantitative comparison of graph-based models for Internet topology , 1997, TNET.

[24]  BERNARD M. WAXMAN,et al.  Routing of multipoint connections , 1988, IEEE J. Sel. Areas Commun..

[25]  Sandeep K. Singhal,et al.  InVerse: Designing an interactive universe architecture for scalability and extensibility , 1997, Proceedings. The Sixth IEEE International Symposium on High Performance Distributed Computing (Cat. No.97TB100183).

[26]  Bruce S. Davie,et al.  Computer Networks: A System Approach , 1998, IEEE Communications Magazine.

[27]  Sandeep K. Singhal,et al.  Exploiting Position History for Efficient Remote Rendering in Networked Virtual Reality , 1995, Presence: Teleoperators & Virtual Environments.

[28]  Grenville Armitage,et al.  A synthetic traffic model for Half-Life , 2003 .

[29]  Eryk Dutkiewicz,et al.  An analysis of multi-player network games traffic , 1999, 1999 IEEE Third Workshop on Multimedia Signal Processing (Cat. No.99TH8451).

[30]  Tim Brecht,et al.  Efficient operating system support for group unicast , 2005, NOSSDAV '05.

[31]  Michael Zyda,et al.  NPSNET:A Network Software Architecture for LargeScale Virtual Environments , 1994, Presence: Teleoperators & Virtual Environments.

[32]  Johannes Färber,et al.  Network game traffic modelling , 2002, NetGames '02.

[33]  Maja Matijasevic,et al.  A Review of Networked Multi-User Virtual Environments , 1997 .

[34]  M. Brandeau,et al.  An overview of representative problems in location research , 1989 .

[35]  Rynson W. H. Lau,et al.  A multi-server architecture for distributed virtual walkthrough , 2002, VRST '02.

[36]  Gary Yeung,et al.  Characterization of user behavior in a multi-player online game , 2005, ACE '05.

[37]  Carey L. Williamson,et al.  Characterizing and modeling user mobility in a cellular data network , 2005, PE-WASUN '05.

[38]  Emmanuel Frécon,et al.  DIVE: a scaleable network architecture for distributed virtual environments , 1998, Distributed Syst. Eng..

[39]  Mostafa H. Ammar,et al.  An evaluation of grouping techniques for state dissemination in networked multi-user games , 2001, MASCOTS 2001, Proceedings Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunication Systems.

[40]  Wu-chi Feng,et al.  Provisioning on-line games: a traffic analysis of a busy counter-strike server , 2002, Comput. Commun. Rev..

[41]  Lixia Zhang,et al.  On the placement of Internet instrumentation , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[42]  Lars C. Wolf,et al.  On the impact of delay on real-time multiplayer games , 2002, NOSSDAV '02.

[43]  O. Kariv,et al.  An Algorithmic Approach to Network Location Problems. I: The p-Centers , 1979 .

[44]  Christophe Diot,et al.  Design and evaluation of MiMaze a multi-player game on the Internet , 1998, Proceedings. IEEE International Conference on Multimedia Computing and Systems (Cat. No.98TB100241).

[45]  Michael S. Wathen Dynamic Scalable Network Area of interest management for Virtual Worlds , 2001 .

[46]  Mark Claypool,et al.  Network analysis of Counter-strike and Starcraft , 2003, Conference Proceedings of the 2003 IEEE International Performance, Computing, and Communications Conference, 2003..

[47]  Angelos Bilas,et al.  Parallelization and performance of interactive multiplayer game servers , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[48]  Timothy J. Lowe,et al.  Convex Location Problems on Tree Networks , 1976, Oper. Res..

[49]  Bo Li,et al.  Server replication and its placement for reliable multicast , 2000, Proceedings Ninth International Conference on Computer Communications and Networks (Cat.No.00EX440).

[50]  Michael Zyda,et al.  Exploiting reality with multicast groups , 1995, IEEE Computer Graphics and Applications.

[51]  John C. S. Lui,et al.  Balancing Workload and Communication Cost for a Distributed Virtual Environment , 1998, Multimedia Information Systems.

[52]  Lili Qiu,et al.  On the placement of Web server replicas , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[53]  Donghoon Lee,et al.  A load balancing algorithm for a distributed multimedia game server architecture , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[54]  Bhaba R. Sarker,et al.  Discrete location theory , 1991 .

[55]  Arthur R Pope,et al.  The SIMNET Network and Protocols. Revision , 1991 .

[56]  Christophe Diot,et al.  A distributed architecture for multiplayer interactive applications on the Internet , 1999, IEEE Netw..

[57]  Mostafa A. Bassiouni,et al.  Performance and reliability analysis of relevance filtering for scalable distributed interactive simulation , 1997, TOMC.

[58]  Yasushi Saito,et al.  Optimistic replication , 2005, CSUR.

[59]  Long Cheng,et al.  System-performance modeling for massively multiplayer online role-playing games , 2006, IBM Syst. J..

[60]  David C. Brogan,et al.  Diamond Park and Spline: a Social Virtual Reality System with 3d Animation, Spoken Interaction, and Runtime Modiiability , 1996 .

[61]  Leonard Kleinrock,et al.  Queueing Systems: Volume I-Theory , 1975 .

[62]  Ronald M. Baecker,et al.  Readings in Groupware and Computer-Supported Cooperative Work: Assisting Human-Human Collaboration , 1992 .

[63]  James M. Calvin,et al.  The SIMNET virtual world architecture , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[64]  Michael Zyda,et al.  Networked virtual environments - desgin and implementation , 1999 .