Using Camera-Phones to Enhance Human–Computer Interaction

The proliferation of camera-phone technology provides a global opportunity for novel ubiquitous computing applications that is, as yet, largely unexploited. Our work demonstrates some of the untapped potential of this existing infrastructure, focusing on the use of deployed camera-phones as devices to enhance human-computer interaction. We show that, without requiring additional hardware, an existing camera phone can be used as both a sophisticated pointing-device, facilitating interaction with active displays; and a user interface for devices without displays or input capability of their own. Our implementation relies on visual tags, which can be detected and decoded by camera-phones, and short-range wireless communication between camera-phones and nearby computers (e.g. Bluetooth). HIGH-LEVEL SYSTEM OVERVIEW Our system relies on two core technologies: 1. Visual tags (known as SpotCodes), that can be detected by camera-phones, are used to encode data. Note that the tags may be either active (e.g. generated dynamically by a PC and written to a display) or passive (e.g. printed on a poster). 2. Bluetooth is used for short-range wireless communication between a camera-phone and a nearby computer. Each visual tag encodes two pieces of information: a service-identifier and a data-block . The service-identifier tells the phone which nearby Bluetooth service the tag relates to; the data-block encodes a few bits of applicationspecific information. We have implemented robust, highperformance tag-recognition software for a number of Figure 1. Data flow through our system