Gesture-Based, Touch-Free Multi-User Gaming on Wall-Sized, High-Resolution Tiled Displays

Having to carry input devices can be inconvenient when interacting with wall-sized, high-resolution tiled displays. Such displays are typically driven by a cluster of computers. Running existing games on a cluster is non-trivial, and the performance attained using software solutions like Chromium is not good enough. This paper presents a touch-free, multi-user, humancomputer interface for wall-sized displays that enables completely device-free interaction. The interface is built using 16 cameras and a cluster of computers, and is integrated with the games Quake 3 Arena (Q3A) and Homeworld. The two games were parallelized using two different approaches in order to run on a 7x4 tile, 21 megapixel display wall with good performance. The touch-free interface enables interaction with a latency of 116 ms, where 81 ms are due to the camera hardware. The rendering performance of the games is compared to their sequential counterparts running on the display wall using Chromium. Parallel Q3A’s framerate is an order of magnitude higher compared to using Chromium. The parallel version of Home

[1]  Grenville J. Armitage,et al.  An experimental estimation of latency sensitivity in multiplayer Quake 3 , 2003, The 11th IEEE International Conference on Networks, 2003. ICON2003..

[2]  Saul Greenberg,et al.  Multimodal multiplayer tabletop gaming , 2007, CIE.

[3]  Adam Finkelstein,et al.  Building and Using A Scalable Display Wall System , 2000, IEEE Computer Graphics and Applications.

[4]  Kajal T. Claypool,et al.  The effects of frame rate and resolution on users playing first person shooter games , 2006, Electronic Imaging.

[5]  Vladimir Pavlovic,et al.  Speech/Gesture Interface to a Visual-Computing Environment , 2000, IEEE Computer Graphics and Applications.

[6]  Greg Humphreys,et al.  Chromium: a stream-processing framework for interactive rendering on clusters , 2002, SIGGRAPH.

[7]  Darren Leigh,et al.  DiamondTouch: a multi-user touch technology , 2001, UIST '01.

[8]  J William,et al.  IEEE Computer Graphics and Applications , 2019, Computer.

[9]  Jefferson Y. Han.  Low-cost multi-touch sensing through frustrated total internal reflection , 2005, UIST.

[10]  I. Scott MacKenzie,et al.  Lag as a determinant of human performance in interactive systems , 1993, INTERCHI.

[11]  Gerald D. Morrison.  A camera-based input device for large interactive displays , 2005, IEEE Computer Graphics and Applications.

[12]  Paul Wielinga,et al.  Building a 100 Mpixel graphics device for the OptIPuter , 2006, Future Gener. Comput. Syst..

[13]  Kristopher J. Blom,et al.  A New Gaming Device and Interaction Method for a First-Person-Shooter , 2005 .

[14]  Greg Burns,et al.  LAM: An Open Cluster Environment for MPI , 2002 .

[15]  Jeffrey Jacobson,et al.  Unreal tournament for immersive interactive theater , 2002, CACM.

[16]  John Markus Bjørndalen,et al.  Lessons Learned Using a Camera Cluster to Detect and Locate Objects , 2007, PARCO.

[17]  Keechul Jung,et al.  Recognition-based gesture spotting in video games , 2004, Pattern Recognit. Lett..

[18]  D MorrisonGerald.  A Camera-Based Input Device for Large Interactive Displays , 2005 .

[19]  Jakub Segen,et al.  Gesture VR: vision-based 3D hand interace for spatial interaction , 1998, MULTIMEDIA '98.