Masking the effects of delays in human-to-human remote interaction

Humans can interact remotely with each other through computers. Systems supporting this include teleconferencing, games and virtual environments. There are delays from when a human does an action until it is reflected remotely. When delays are too large, they will result in inconsistencies in what the state of the interaction is as seen by each participant. The delays can be reduced, but they cannot be removed. When delays become too large the effects they create on the human-to-human remote interaction can be partially masked to achieve an illusion of insignificant delays. The MultiStage system is a human-to-human interaction system meant to be used by actors at remote stages creating a common virtual stage. Each actor is remotely represented by a remote presence created based on a stream of data continuously recorded about the actor and being sent to all stages. We in particular report on the subsystem of MultiStage masking the effects of delays. The most advanced masking approach is done by having each stage continuously look for late data, and when masking is determined to be needed, the system switches from using a live stream to a pre-recorded video of an actor. The system can also use a computable model of an actor creating a remote presence substituting for the live stream. The present prototype uses a simple human skeleton model.

[1]  Paul Francis,et al.  IDMaps: a global internet host distance estimation service , 2001, TNET.

[2]  Yutaka Ishibashi,et al.  Influences of network latency and packet loss on consistency in networked racing games , 2005, NetGames '05.

[3]  Lars C. Wolf,et al.  Analysis of factors affecting players' performance and perception in multiplayer games , 2005, NetGames '05.

[4]  Arun Venkataramani,et al.  A structural approach to latency prediction , 2006, IMC '06.

[5]  Yutaka Ishibashi,et al.  Interactivity improvement of group synchronization control in collaborative haptic play with building blocks , 2010, 2010 9th Annual Workshop on Network and Systems Support for Games.

[6]  Katerina Mania,et al.  A system to measure, control and minimize end-to-end head tracking latency in immersive simulations , 2011, VRCAI '11.

[7]  John Markus Bjørndalen,et al.  MultiStage: Acting across Distance , 2013, ECLAP.

[8]  Tolga K. Çapin,et al.  A dead-reckoning algorithm for virtual human figures , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[9]  Lars C. Wolf,et al.  On the impact of delay on real-time multiplayer games , 2002, NOSSDAV '02.

[10]  Krishna P. Gummadi,et al.  King: estimating latency between arbitrary internet end hosts , 2002, IMW '02.

[11]  Kyle Johnsen,et al.  Ubiquitous collaborative activity virtual environments , 2012, CSCW '12.

[12]  Thomas Malzbender,et al.  Understanding performance in coliseum, an immersive videoconferencing system , 2005, TOMCCAP.

[13]  Elaine Chew,et al.  Distributed musical performances: Architecture and stream management , 2008, TOMCCAP.

[14]  Sampath Rangarajan,et al.  Fairness in dead-reckoning based distributed multi-player games , 2005, NetGames '05.

[15]  Antoine Beugnard,et al.  A dynamic approach to consistency management for mobile multiplayer games , 2008, NOTERE.

[17]  Sampath Rangarajan,et al.  Accuracy in dead-reckoning based distributed multi-player games , 2004, NetGames '04.

[18]  Grenville J. Armitage,et al.  Limitations of using real-world, public servers to estimate jitter tolerance of first person shooter games , 2004, ACE '04.

[19]  Kajal T. Claypool,et al.  Latency and player actions in online games , 2006, CACM.

[20]  Yi Zhang,et al.  Globally synchronized dead-reckoning with local lag for continuous distributed multiplayer games , 2006, NetGames '06.

[21]  Andriy Pavlovych,et al.  Target following performance in the presence of latency, jitter, and signal dropouts , 2011, Graphics Interface.

[22]  Carl Gutwin,et al.  Assessing target acquisition and tracking performance for complex moving targets in the presence of latency and jitter , 2012, Graphics Interface.

[23]  Stephen John Turner,et al.  Fair and Efficient Dead Reckoning-Based Update Dissemination for Distributed Virtual Environments , 2012, 2012 ACM/IEEE/SCS 26th Workshop on Principles of Advanced and Distributed Simulation.

[24]  Robin Wolff,et al.  Comparing the End to End Latency of an Immersive Collaborative Environment and a Video Conference , 2009, 2009 13th IEEE/ACM International Symposium on Distributed Simulation and Real Time Applications.

[25]  Vasily Y. Kharitonov,et al.  Motion-aware adaptive dead reckoning algorithm for collaborative virtual environments , 2012, VRCAI '12.

[26]  John C. Tang,et al.  Three's company: understanding communication channels in three-way distributed collaboration , 2010, CSCW '10.

[27]  Michael Bredel,et al.  A measurement study regarding quality of service and its impact on multiplayer online games , 2010, 2010 9th Annual Workshop on Network and Systems Support for Games.

[28]  Xinbo Jiang,et al.  Latency and scalability: a survey of issues and techniques for supporting networked games , 2005, 2005 13th IEEE International Conference on Networks Jointly held with the 2005 IEEE 7th Malaysia International Conf on Communic.

[29]  Yahn W. Bernier,et al.  Latency Compensating Methods in Client/Server In-game Protocol Design and Optimization , 2003 .

[30]  Jitendra Padhye,et al.  Measurement and Estimation of Network QoS Among Peer Xbox 360 Game Players , 2008, PAM.

[31]  A. A. Sawchuk,et al.  From remote media immersion to Distributed Immersive Performance , 2003, ETP '03.