Point-and-shoot for ubiquitous tagging on mobile phones

We propose a novel way to augment a real scene with minimalist user intervention on a mobile phone: The user only has to point the phone camera to the desired location of the augmentation. Our method is valid for vertical or horizontal surfaces only, but this is not a restriction in practice in man-made environments, and avoids to go through any reconstruction of the 3D scene, which is still a delicate process. Our approach is inspired by recent work on perspective patch recognition [5] and we show how to modify it for better performances on mobile phones and how to exploit the phone accelerometers to relax the need for fronto-parallel views. In addition, our implementation allows to share the augmentations and the required data over peer-to-peer communication to build a shared AR space on mobile phones.

[1]  Tom Drummond,et al.  Multiple Target Localisation at over 100 FPS , 2009, BMVC.

[2]  Jitendra Malik,et al.  Geometric blur for template matching , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[3]  Selim Benhimane,et al.  Homography-based 2D Visual Tracking and Servoing , 2007, Int. J. Robotics Res..

[4]  Pished Bunnun,et al.  OutlinAR: an assisted interactive model building system with reduced computational effort , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[5]  Rudolph Ernest Langer,et al.  Relativity And Modern Physics , 1925 .

[6]  David W. Murray,et al.  Improving the Agility of Keyframe-Based SLAM , 2008, ECCV.

[7]  Gilles Simon.  Immersive image-based modeling of polyhedral scenes , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[8]  Stephen DiVerdi,et al.  "Anywhere Augmentation": Towards Mobile Augmented Reality in Unprepared Environments , 2007, Location Based Services and TeleCartography.

[9]  Tom Drummond,et al.  ProFORMA: Probabilistic Feature-based On-line Rapid Model Acquisition , 2009, BMVC.

[10]  David W. Murray,et al.  Parallel Tracking and Mapping on a camera phone , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[11]  Nassir Navab,et al.  A dataset and evaluation methodology for template-based tracking algorithms , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[12]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[13]  Valerio Faraoni,et al.  Cosmology in Scalar-Tensor Gravity , 2004 .

[14]  Jeremiah Neubert,et al.  Semi-Autonomous Generation of Appearance-based Edge Models from Image Sequences , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[15]  Michel Dhome,et al.  Hyperplane Approximation for Template Matching , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Vincent Lepetit,et al.  Noname manuscript No. (will be inserted by the editor) Learning Real-Time Perspective Patch Rectification , 2022 .

[17]  Dieter Schmalstieg,et al.  Multiple target detection and tracking with guaranteed framerates on mobile phones , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[18]  Vincent Lepetit,et al.  ESM-Blur: Handling & rendering blur in 3D tracking and augmentation , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[19]  Natasha Gelfand,et al.  SURFTrac: Efficient tracking and continuous object recognition using local feature descriptors , 2009, CVPR.

[20]  Dieter Schmalstieg,et al.  Pose tracking from natural features on mobile phones , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.