Integrating 2-D Video Actors into 3-D Augmented-Reality Systems

In this paper, we discuss the integration of 2-D video actors into 3-D augmentedreality (AR) systems. In the context of our research on narrative forms for AR, we have found ourselves needing highly expressive content that is most easily created by human actors. We discuss the feasibility and utility of using video actors in an AR situation and then present our Video Actor Framework (including the VideoActor editor and the Video3D Java package) for easily integrating 2-D videos of actors into Java 3D, an object-oriented 3-D graphics programming environment. The framework is based on the idea of supporting tight spatial and temporal synchronization between the content of the video and the rest of the 3-D world. We present a number of illustrative examples that demonstrate the utility of the toolkit and editor. We close with a discussion and example of our recent work implementing these ideas in Macromedia Director, a popular multimedia production tool.

[1]  J. Bolter,et al.  Remediation: Understanding New Media , 1999 .

[2]  Robert DeLine,et al.  Alice: Rapid prototyping system for virtual reality , 1995 .

[3]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[4]  David C. Brogan,et al.  Animating human athletics , 1995, SIGGRAPH.

[5]  Steven K. Feiner,et al.  Augmented Reality in Architectural Construction, Inspection, and Renovation , 1996 .

[6]  J. Robert Ensor,et al.  Techniques for handling video in virtual environments , 1998, SIGGRAPH.

[7]  Richard Szeliski,et al.  The lumigraph , 1996, SIGGRAPH.

[8]  Mary C. Whitton,et al.  Technologies for augmented reality systems: realizing ultrasound-guided needle biopsies , 1996, SIGGRAPH.

[9]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[10]  John Lasseter,et al.  Principles of traditional animation applied to 3D computer animation , 1987, SIGGRAPH.

[11]  Richard Szeliski,et al.  Video textures , 2000, SIGGRAPH.

[12]  Ralf Dörner,et al.  Performing 3D scene and animation authoring tasks efficiently: an innovative approach , 1998, Proceedings. Computer Graphics International (Cat. No.98EX149).

[13]  Klaus H. Hinrichs,et al.  Interactive, animated 3D widgets , 1998, Proceedings. Computer Graphics International (Cat. No.98EX149).

[14]  Verna Allee,et al.  Chapter 3 – The Knowledge Navigators , 1997 .

[15]  Jay David Bolter,et al.  Alice's Adventure's in New Media: An Exploration of Interactive Narratives in Augmented Reality , 2001 .

[16]  Blair MacIntyre,et al.  Augmented reality as a new media experience , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[17]  John Lasseter,et al.  Principles of traditional animation applied to 3D computer animation , 1987, SIGGRAPH.

[18]  Steven K. Feiner,et al.  Architectural Anatomy , 1995, Presence: Teleoperators & Virtual Environments.

[19]  Amy Wohl,et al.  Apple computer, Inc. , 2003 .

[20]  Robert Taylor,et al.  Disney's Aladdin: first steps toward storytelling in virtual reality , 1996, SIGGRAPH.