Single- and Dual-User Web Browsing in the Transparent Video Facetop

The Transparent Video Facetop is a novel user interface concept that supports not only single-user interactions with a PC, but also close pair collaborations, such as that found in collaborative Web browsing, in distributed pair programming and in distributed extreme programming (dPP/dXP). We recently demonstrated the Vis-a-Vid facetop prototype as a single-user GUI for manipulating the elements of a traditional WIMP desktop [21]. In this paper we show how the single-user transparent video facetop can be used for fingertip control of a Web browser, and how a dual-head facetop can be used for paired synchronous (collaborative) Web browsing. The facetop is not a new browser, per se, but rather a novel way to interact with a Web brower, as well as a novel means of providing visual interaction among users collaboratively browsing the Web. BASIC FACETOP CONCEPTS The transparent video facetop is a novel enhancement of the traditional WIMP user interface, so nearly ubiquitous on today’s computers. In the facetop, the user sees him/her self as a “ghostly” image apparently behind the desktop, looking back at the icons and windows from the back. Instead of a traditional desktop, we see a “face” top. This self-image is used for visual feedback and communications both to the user as well as to collaborators; it is also used for desktop/application control and manipulation via a fingertipdriven “virtual mouse”. Figure 1: Facetop physical setup, with iBot video camera Figure 1 shows the physical setup for a computer with a facetop being displayed on a monitor. Note the video camera sitting on top the LCD panel pointing back at the user; in our current work we use a $100 Sony iBot, giving us an image that is 640 x 480 pixels of 24-bit color, captured 30 frames per second. The facetop video window shows the PC user sitting at his/her workspace; we reverse the image horizontally so that when the user moves a hand, say, to the left, the image of the hand mirrors this movement on the screen. In software, and using a high-performance 3D-graphics video card, we make the video window semi-transparent and composite it with the desktop image itself. Once we have the full screen video with transparent image compositing we get the illusion of the user watching the desktop from behind. Mirroring means if the user physically points to an icon on the desktop, the facetop image points to the icon as well (with proper spatial calibration of the camera and user locations). Using image analysis techniques we then track the user’s fingertip in the backing window, and optionally drive the mouse from this tracker. The user can then manipulate the desktop of a projected computer from his seat while successfully communicating the areas of interest on the screen to others watching the projection. Projected displays vs. monitors The facetop as a concept works fine on a PC with any display technology -a monitor, a projector, an immersive device -but its unique aspects are most pronounced and most effective in a projected environment. The concept of background user video as visual cues for control and communication came about when our research group was discussing other work using a projected PC. We were all sitting in chairs viewing the projection wall, but constantly pointing at the desktop image 5 to 10 feet away. Determining where on the wall/screen to look was largely an exercise in visually interpolating along the line formed by a person’s pointing arm. With the facetop, the user is right there in the desktop image and gives an immediate visual cue as to where to look when pointing. Figure 6 shows the facetop projected.