Ghosts in the Machine: Integrating 2D Video Actors into a 3D AR System

In this paper we discuss the integration of 2D video actors into a 3D augmented reality (AR) system. In the context of our research on narratives forms for AR, we have found ourselves needing highly expressive content that is most easily created by human actors. We discuss the feasibility and utility of using video actors in an AR situation, and then present our Video Actor Framework (including the VideoActor editor and the Video3D Java package) for easily integrating 2D videos of actors into Java3D, an object-oriented 3D graphics programming environment. The framework is based on the idea of supporting tight spatial and temporal synchronization between the content of the video and the rest of the 3D world. We present a number of illustrative examples that demonstrate the utility of the toolkit and editor.

[1]  Kelly L. Murdock.  3D Studio MAX® R3 Bible , 2000 .

[2]  David C. Brogan,et al.  Animating human athletics , 1995, SIGGRAPH.

[3]  J. Robert Ensor,et al.  Techniques for handling video in virtual environments , 1998, SIGGRAPH.

[4]  Ralf Dörner,et al.  Performing 3D scene and animation authoring tasks efficiently: an innovative approach , 1998, Proceedings. Computer Graphics International (Cat. No.98EX149).

[5]  Steven K. Feiner,et al.  Architectural Anatomy , 1995, Presence: Teleoperators & Virtual Environments.

[6]  Robert Taylor,et al.  Disney's Aladdin: first steps toward storytelling in virtual reality , 1996, SIGGRAPH.

[7]  R. Day.  Remediation: Understanding new media , 1999 .

[8]  Klaus H. Hinrichs,et al.  Interactive, animated 3D widgets , 1998, Proceedings. Computer Graphics International (Cat. No.98EX149).

[9]  Mary C. Whitton,et al.  Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies , 1996, SIGGRAPH.

[10]  Steven K. Feiner,et al.  Augmented Reality in Architectural Construction, Inspection, and Renovation , 1996 .

[11]  Robert DeLine,et al.  Alice: Rapid prototyping system for virtual reality , 1995 .

[12]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[13]  Verna Allee,et al.  Chapter 3 – The Knowledge Navigators , 1997 .

[14]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  John Lasseter,et al.  Principles of traditional animation applied to 3D computer animation , 1987, SIGGRAPH.