When Worlds Collide - Interactions Between the Virtual and the Real

This paper explores issues surrounding interaction with virtual (computer generated) objects which are interfaced to real world devices. In addition a virtual room corresponding to a real physical room has been created to allow collaborative meetings with those physically and virtually present. In particular we have created a set of VRML (Virtual Reality Modeling Language) cameras, which interface to real computer controlled cameras in a " smart " room. Interacting with the virtual camera controls the real camera. A second interaction method is through a Java applet, which appears on a Web page as an image of the remote control for the camera. The VRML camera object can also function as a status display. When someone moves the camera in the real world via the real remote control, the position of the VRML camera updates to reflect the actual status. We have two methods of viewing the video from the camera, one a C program executing on the CPU with the camera which quickly updates the images that can be viewed on the web via a server-push HTML page. The second method is to stream the video through video streaming software (commercial off-the-shelf). The entire VRML view of the " smart " room with cameras is placed within the context of a multiuser VRML world. This world can be visited by whomever desires to participate in meetings, held in the physical " smart " room. Our intent is to create intuitive user interfaces to the cameras, and other facilities of the room. We allow people not physically able or willing to attend the physical meeting to participate " virtually ". People attending meetings remotely are represented as avatars. The room and objects in it are used for control and status. Depending on which, the dialog between the user and the displays change. When controlling real devices feedback must be rapid and sufficient to allow meaningful control. When using the virtual devices of the entire room only for status the user is more passive and can focus on other aspects of the meeting such as the actual content of the talk while still getting a feeling of the physical nature of the meeting space. 1 BACKGROUND AND RELATED WORK The Virtual Reality Modeling Language (VRML) is a recently ratified ISO standard (ISO 14772) [8] file format for the description of geometry and behavior of 3D computer graphics. VRML is the most …