Location-based augmented reality on mobile phones

The computational capability of mobile phones has been rapidly increasing, to the point where augmented reality has become feasible on cell phones. We present an approach to indoor localization and pose estimation in order to support augmented reality applications on a mobile phone platform. Using the embedded camera, the application localizes the device in a familiar environment and determines its orientation. Once the 6 DOF pose is determined, 3D virtual objects from a database can be projected into the image and displayed for the mobile user. Off-line data acquisition consists of acquiring images at different locations in the environment. The online pose estimation is done by a feature-based matching between the cell phone image and an image selected from the precomputed database using the phone's sensors (accelerometer and magnetometer). The application enables the user both to visualize virtual objects in the camera image and to localize the user in a familiar environment. We describe in detail the process of building the database and the pose estimation algorithm used on the mobile phone. We evaluate the algorithm performance as well as its accuracy in terms of reprojection distance of the 3D virtual objects in the cell phone image.