Vision-based all-in-one solution for augmented reality and its storytelling applications

In this paper, we propose a vision-based all-in-one solution for augmented reality where users want to exploit unknown 3D objects in their systems. In our solution, we facilitate two time-consuming off-line processes: obtaining information, such as keyframes and keypoints, for real-time tracking of unknown 3D targets, and estimating local coordinates with a scale for accurate registration of virtual content. The proposed solution only requires images with minimal interactions. The users do not need to know about 3D markerless tracking in depth. At the end, we propose a framework for AR miniatures systems to verify the effectiveness of our solution. As a result, all developed systems worked in real-time, more than 25 fps, and showed reliable registration even in severe viewpoint changes. Our demonstration videos are available in the supplemental materials.

[1]  Jan Mayer,et al.  A numerical evaluation of preprocessing and ILU-type preconditioners for the solution of unsymmetric sparse linear systems using iterative methods , 2009, TOMS.

[2]  Vincent Lepetit,et al.  Scalable real-time planar targets tracking for digilog books , 2010, The Visual Computer.

[3]  Horst Bischof,et al.  From structure-from-motion point clouds to fast location recognition , 2009, CVPR.

[4]  Shogo Nishida,et al.  Virtual Pop-Up Book Based on Augmented Reality , 2007, HCI.

[5]  Youngmin Park,et al.  Digilog miniature: real-time, immersive, and interactive AR on miniatures , 2010, VRCAI '10.

[6]  Thomas B. Moeslund,et al.  Long-Term Occupancy Analysis Using Graph-Based Optimisation in Thermal Imagery , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Dieter Schmalstieg,et al.  ARToolKitPlus for Pose Trackin on Mobile Devices , 2007 .

[8]  Anton van den Hengel,et al.  In situ image-based modeling , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[9]  Miika Aittala,et al.  Inverse lighting and photorealistic rendering for augmented reality , 2010, The Visual Computer.

[10]  David W. Murray,et al.  Object recognition and localization while tracking and mapping , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[11]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[12]  Vincent Lepetit,et al.  An all-in-one solution to geometric and photometric calibration , 2006, 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Anton van den Hengel,et al.  Interactive modelling for AR applications , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[15]  Jan-Michael Frahm,et al.  Piecewise planar and non-planar stereo for urban scene reconstruction , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  David W. Murray,et al.  Keyframe-based recognition and localization during video-rate parallel tracking and mapping , 2011, Image Vis. Comput..

[17]  Vincent Lepetit,et al.  Randomized trees for real-time keypoint recognition , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[18]  Vincent Lepetit,et al.  Accurate Non-Iterative O(n) Solution to the PnP Problem , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[19]  David G. Lowe,et al.  Scene modelling, recognition and tracking with invariant image features , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[20]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[21]  Mark Fiala,et al.  ARTag, a fiducial marker system using digital techniques , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Manolis I. A. Lourakis,et al.  SBA: A software package for generic sparse bundle adjustment , 2009, TOMS.

[23]  Hujun Bao,et al.  Keyframe-based real-time camera tracking , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[24]  Vincent Lepetit,et al.  The haunted book , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[25]  Andrew J. Davison,et al.  Live dense reconstruction with a single moving camera , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Jan-Michael Frahm,et al.  From structure-from-motion point clouds to fast location recognition , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Nuno Correia,et al.  Storing user experiences in mixed reality using hypermedia , 2006, The Visual Computer.

[28]  S. Poovizhi,et al.  An Implementation of Scale Invariant Feature Transform (SIFT) Algorithm Using Content Based Image Retrieval , 2014 .

[29]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[30]  Pished Bunnun,et al.  OutlinAR: an assisted interactive model building system with reduced computational effort , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[31]  George Papagiannakis,et al.  Presence and interaction in mixed reality environments , 2007, The Visual Computer.

[32]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.