Tangible interfaces for real-time 3D virtual environments

Emergent game formats, such as machinima, that use game worlds as expressive 3D performance spaces have new expressive powers with an increase of the quality of their underlying graphic and animation systems. Nevertheless, they still lack intuitive control mechanisms. Set direction and acting are limited by tools that were designed to create and play video games rather than produce expressive performance pieces. These tools do a poor job of capturing the performative expression that characterizes more mature media such as film. Tangible interfaces can help open up the game systems for more intuitive character control needed for a greater level of expression in the digital real-time world. The TUI3D project (Tangible User Interfaces for Real-Time 3D) addresses production and performative challenges involved in creating machinima through the development of tangible interfaces for controlling 3D virtual actors and environments in real-time. In this paper, we present the design and implementation of a tangible puppet prototype for virtual character control in the Unreal game engine and discuss initial user feedback.

[1]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[2]  Brenda Laurel,et al.  Computers as theatre , 1991 .

[3]  Chris Esposito,et al.  Of mice and monkeys: a specialized input device for virtual body animation , 1995, I3D '95.

[4]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[5]  Bruce Blumberg,et al.  Sympathetic interfaces: using a plush toy to direct synthetic characters , 1999, CHI '99.

[6]  Erik Strommen,et al.  When the interface is a talking dinosaur: learning across media with ActiMates Barney , 1998, CHI.

[7]  Lucien King.  Game on: The History and Culture of Video Games , 2002 .

[8]  Noah Wardrip-Fruin,et al.  First Person: New Media As Story, Performance, And Game , 2004 .

[9]  Jesper Juul.  Half-Real: Video Games between Real Rules and Fictional Worlds , 2005 .

[10]  John M. Carroll,et al.  Human-Computer Interaction in the New Millennium , 2001 .

[11]  Robert J. K. Jacob,et al.  The perceptual structure of multidimensional input device selection , 1992, CHI.

[12]  Ronald A. Metoyer,et al.  A Tangible Interface for High-Level Direction of Multiple Animated Characters , 2003, Graphics Interface.

[13]  Hiroshi Ishii,et al.  Emerging frameworks for tangible user interfaces , 2000, IBM Syst. J..

[14]  J. Murray.  Hamlet on the Holodeck , 1997 .

[15]  David Zeltzer,et al.  A design method for “whole-hand” human-computer interaction , 1993, TOIS.

[16]  Katie Salen,et al.  Rules of play: game design fundamentals , 2003 .

[17]  Jean-Michel Michenaud,et al.  The making of Jurassic Park , 1995 .

[18]  M. Resnick,et al.  Programmable Bricks: Toys to Think With , 1996, IBM Syst. J..

[19]  Simon Byron,et al.  Game On , 2006 .

[20]  Paul Marino,et al.  3D geme-based filmmaking : the art of machinima , 2004 .

[21]  Randy Nichols.  Review: Jesper Juul's Half-Real: Video Games Between Real Rules and Fictional Worlds , 2006 .

[22]  Michael Nitsche,et al.  Puppet Show : Intuitive Puppet Interfaces for Expressive Character Control , 2006 .