Mixed Reality Tabletop (MRT): A Low-Cost Teleconferencing Framework for Mixed-Reality Applications

Today’s technology enables a rich set of virtual and mixedreality applications and provides a degree of connectivity and interactivity beyond that of traditional teleconferencing scenarios. In this paper, we present the Mixed-Reality Tabletop (MRT), an example of a teleconferencing framework for networked mixed reality in which real and virtual worlds coexist and users can focus on the task instead of computer interaction. For example, students could use real-world objects to participate in physical simulations such as orbital motion, collisions, and fluid flow in a common virtual environment. Our framework isolates the lowlevel system details from the developer and provides a simple programming interface for developing novel applications in as little as a few minutes, using low-cost hardware. We discuss our implementation complete with methods of hand and object tracking, user interface, and example applications focused on remote teaching and learning.

[1]  Christian Breiteneder,et al.  TELEPORT—an augmented reality teleconferencing environment , 1996 .

[2]  Hideki Koike,et al.  EnhancedDesk: integrating paper documents and digital documents , 1998, Proceedings. 3rd Asia Pacific Computer Human Interaction (Cat. No.98EX110).

[3]  Hideyuki Tamura,et al.  MR Platform: a basic body on which mixed reality applications are built , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[4]  Bruce H. Thomas,et al.  An object-oriented software architecture for 3D mixed reality applications , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[5]  Andrew D. Wilson.  TouchLight: an imaging touch screen and display for gesture-based interaction , 2004, ICMI '04.

[6]  Björn Stenger,et al.  Filtering using a tree-based estimator , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[7]  Markus H. Gross,et al.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[8]  Claudio S. Pinhanez.  The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces , 2001, UbiComp.

[9]  Steven K. Feiner,et al.  Collaborative mixed reality visualization of an archaeological excavation , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[10]  William Ribarsky,et al.  The Perceptive Workbench: toward spontaneous and natural interaction in semi-immersive virtual environments , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[11]  Tom Drummond,et al.  Sensor fusion and occlusion refinement for tablet-based AR , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[12]  Rong Xiao,et al.  Boosting chain learning for object detection , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[13]  William Ribarsky,et al.  T owards Spontaneous Interaction with the Perceptive Workbench, a Semi-Immersive Virtual Environment* , 2000 .

[14]  Jianbo Shi,et al.  Tele-Graffiti: A Camera-Projector Based Remote Sketching System with Hand-Based User Interface and Automatic Session Summarization , 2003, International Journal of Computer Vision.

[15]  Hiroshi Ishii,et al.  Emancipated pixels: real-world graphics in the luminous room , 1999, SIGGRAPH.

[16]  Hiroshi Ishii,et al.  Augmented urban planning workbench: overlaying drawings, physical models and digital simulation , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[17]  Luc Van Gool,et al.  Blue-c: a spatially immersive display and 3D video portal for telepresence , 2003, IPT/EGVE.

[18]  Morten Fjeld,et al.  BUILD-IT: A Computer Vision-based Interaction Technique for a Planning Tool , 1997, BCS HCI.

[19]  Hiroshi Ishii,et al.  The metaDESK: models and prototypes for tangible user interfaces , 1997, UIST '97.

[20]  Thomas Furness,et al.  Toward directly mediated interaction in computer supported environments , 2004 .