What is That? Object Recognition from Natural Features on a Mobile Phone

Connecting the physical and the digital world is an upcoming trend that enables numberless use-cases. The mobile phone as the most pervasive digital device is often used to establish such a connection. The phone enables users to retrieve, use, and share digital information and services connected to physical objects. Recognizing physical objects is thus a fundamental precondition for such applications. Object recognition is usually enabled using visual marker (e.g. Qr Codes) or electronic marker (e.g. RFID). Marker based approaches are not feasible for a large range of objects such as sights, photos, and persons. Markerless approaches that use the image stream from the mobile phone’s camera are commonly server-based which dramatically limits the interactiveness. Recent work on image processing shows that interactive object recognition on mobile phones is at hand. In this paper we present a markerless object recognition that processes multiple camera images per second on recent mobile phones. The algorithm combines a stripped down SIFT with a scalable vocabulary tree and a simple feature matching. Based on this algorithm we implemented a simple application which recognizes poster segments and conducted an initial user study to get an understanding of the implications that accompany markerless interaction.