Tangible linker: Organizing and accessing digital contents using untagged tangible objects

Retrieving digital contents consists in some searching operation, which requires some memory effort and can be challenging due to the large amount of contents stored by users. Bookmarks and shortcuts are commonly used to ease the process however in a short term they show the same inconveniences above. We propose a novel solution based on tangible media for mapping multiple digital contents using untagged physical objects. Users can map different kinds of related digital contents, regardless their sources, with physical objects and access directly the desired contents without requiring any search operation. Objects in our system require no modifications; instead we use computer vision for object detection and recognition using a conventional web camera. In our implementation, when an object is placed in front of the screen it becomes part of the computer desktop and can be used to retrieve information. We performed an evaluation experiment and results of the evaluation showed that most users found the system easy to use and the interface intuitive. Introduction Modern personal information management (Lansdale, 1988) occurs frequently in computing environments. In such environments, to retrieve digital contents users must reach the content by means of some searching operation, which requires memory effort and turns challenging due to the large amount of contents required for daily work. The conventional solution is keeping bookmarks and shortcuts icons for relevant contents under the WIMP (windows, icons, menu and pointing device) paradigm. This paradigm had gained a remarkable degree of commonality. However, due to the rapid evolution of computer devices, demand for novel interaction paradigms has also increased. One of such new interaction approaches is by using tangible interfaces, which enables users to interact with computers using physical objects. It is in human nature the skill to sense and manipulate the physical environment (Ishii, 2008). Therefore we can take advantage of this ability by giving physical attributes to digital information, making it manageable in the physical domain. Previous studies had defined 3 types of physical objects to represent digital information: container, tokens and tools (Holmquist, 1999). In particular case of token-based interaction, physical objects are used to access stored information and the nature of which is physically reflected in the token in some way. The objects do not contain the information but act as trigger to show the information through a display device. Several previous works employ electronic tagging to link digital information to physical objects. Such tagging could be computer-readable digital tags like barcodes (Ljungstrand, 2000; Siio, 1999) or RFID technology (Want, 1999; Bernheim, 2005; Wellner, 1991). This kind of system allows association mostly of a single content per physical object. In this paper we propose a novel solution based on tangible media for mapping multiple digital contents using untagged physical objects. Since tags are vulnerable to physical damage, deterioration in time and cause visual occlusion to object appearance, we use instead computer vision for object detection and recognition. This way, any physical object around the user can become an interactive object without restrictions as represented in Figure 1. In addition, considering the immediate information needs where user requires a few subsets of the total contents to be readily available, our system allows users to map different kinds of related digital contents, regardless their sources, with physical objects and to access directly the desired contents without requiring any search operation. The main contribution of this work is to empower physical objects around the user, without any modifications, to become an interactive element in a tangible interface approach. To demonstrate the concept we implemented and evaluate a prototype system called Tangible Linker to organize digital contents using physical objects. Figure 1: Objects in the surrounding can represent personal information. Related Work Physical browsing (Valkkynen , 2005) describes the possibility of digital information management through physical objects under an ambient intelligence scenario. IconSticker (Siio, 1999) presents a metaphor of this concept, by giving a paper representation to digital contents. It consists in transferring the icon on user computer desktop to paper, by converting them into barcode representation and printing them out on stickers. These stickers then can be attached to any physical objects. Access to the icon consists in scanning the barcode on the sticker. WebSticker by Ljungstrand et al. (Ljungstrand, 2000) and AURA by Bernheim et al. (Bernheim, 2005) both propose the usage of barcode to represent online information. WebSticker shares the same analogy with IconSticker, while AURA takes advantage of already existing barcode on surrounding objects. They use handheld device with barcode reading function to capture the input and display related information. These mentioned systems are based on visual tagging to identify the object. Other than barcodes, Want et al. (Want, 1999) proposes to augment everyday objects via embedded RFID tag. Compared to previous works, although tags provide higher recognition accuracy, in our system we want to avoid all kinds of computer readable tags and other object modifications to simplify the interaction. Therefore we choose to use computer vision to detect and recognize user objects as they are. Also different from previous works, we combine icons, web links and other digital references into a common launcher, where a single object can represent many references. System Overview We propose a novel method to organize and access digital information using physical objects as references containers. Multiple associations between objects and contents are allowed: an object can be associated to many contents and vice versa. In our system, physical objects do not contain physical memory nor tagged. The associations between digital information and the objects are saved in the computer. The interaction starts when the user places an object in front of the screen (the capturing area). This setup gives the impression of the object being another part of the graphical user interface, seemingly to extend the digital desktop into the physical world and vice versa. To store a reference into an object, the user places the object in front of the screen, as shown in Figure 2(a), and drags the reference toward the direction of the object regarding to the coordinates system of the screen, as shown in Figure 2(b). Next time, when the same object is placed in the capturing area, the system automatically recover the list of references and display them on the screen next to the object, as presented in Figure 2(c). User then can access these references by clicking on the link. In Figure 2(d), the user selected a music list file and it is opened in the default music player application . Once the user finished using the object, removes the object from the capturing area and the interface will close automatically (see Figure 2(e)). Additionally the user can hide or control de opacity of the interface and drop in area marker, delete associations and ask the system to trigger automatically the references. As there is no physical memory on the objects, the capacity is only limited by the computer hard drive.