Going out: robust model-based tracking for outdoor augmented reality

This paper presents a model-based hybrid tracking system for outdoor augmented reality in urban environments enabling accurate, realtime overlays for a handheld device. The system combines several well-known approaches to provide a robust experience that surpasses each of the individual components alone: an edge-based tracker for accurate localisation, gyroscope measurements to deal with fast motions, measurements of gravity and magnetic field to avoid drift, and a back store of reference frames with online frame selection to re-initialize automatically after dynamic occlusions or failures. A novel edge-based tracker dispenses with the conventional edge model, and uses instead a coarse, but textured, 3D model. This yields several advantages: scale-based detail culling is automatic, appearance-based edge signatures can be used to improve matching and the models needed are more commonly available. The accuracy and robustness of the resulting system is demonstrated with comparisons to map-based ground truth data.

[1]  Suya You,et al.  A robust hybrid tracking system for outdoor augmented reality , 2004, IEEE Virtual Reality 2004.

[2]  Christian Früh,et al.  Constructing 3D City Models by Merging Aerial and Ground Views , 2003, IEEE Computer Graphics and Applications.

[3]  Simon J. Julier,et al.  Authoring of physical models using mobile computers , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[4]  Volker Coors,et al.  Matching buildings: pose estimation in an urban environment , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[5]  Tom Drummond,et al.  Sensor fusion and occlusion refinement for tablet-based AR , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[6]  Bruce H. Thomas,et al.  Tinmith-Metro: new outdoor techniques for creating city models with an augmented reality wearable computer , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[7]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Didier Stricker,et al.  Adaptive line tracking with multiple hypotheses for augmented reality , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[9]  Tom Drummond,et al.  Fusing points and lines for high performance tracking , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[10]  Vincent Lepetit,et al.  Combining edge and texture information for real-time accurate 3D camera tracking , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[11]  Suya You,et al.  Approaches to Large-Scale Urban Modeling , 2003, IEEE Computer Graphics and Applications.

[12]  Reinhold Behringer,et al.  Registration for outdoor augmented reality applications using computer vision techniques and hybrid sensors , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[13]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[14]  Suya You,et al.  Tracking with omni-directional vision for outdoor AR systems , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[15]  David G. Lowe,et al.  Scene modelling, recognition and tracking with invariant image features , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[16]  Roberto Cipolla,et al.  Visual tracking and control using Lie algebras , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[17]  Simon J. Julier,et al.  A tracker alignment framework for augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[18]  Bruce H. Thomas,et al.  A wearable computer system with augmented reality to support terrestrial navigation , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[19]  Didier Stricker,et al.  Tracking with reference images: a real-time and markerless tracking solution for out-door augmented reality applications , 2001, VAST '01.

[20]  Tom Drummond,et al.  Robust visual tracking for non-instrumental augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[21]  Hirokazu Kato,et al.  Marker tracking and HMD calibration for a video-based augmented reality conferencing system , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[22]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[23]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[24]  Roberto Cipolla,et al.  An Image-Based System for Urban Navigation , 2004, BMVC.

[25]  Ronald Azuma,et al.  A motion-stabilized outdoor augmented reality system , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[26]  Dieter Schmalstieg,et al.  First steps towards handheld augmented reality , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[27]  Harald Ganster,et al.  Hybrid Tracking for Outdoor Augmented Reality Applications , 2002, IEEE Computer Graphics and Applications.

[28]  Roberto Cipolla,et al.  Building Architectural Models from Many Views Using Map Constraints , 2002, ECCV.

[29]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[30]  Ronald Azuma,et al.  Orientation Tracking for Outdoor Augmented Reality Registration , 1999, IEEE Computer Graphics and Applications.

[31]  Suya You,et al.  Colorplate: A Robust Hybrid Tracking System for Outdoor Augmented Reality , 2004 .

[32]  Steven K. Feiner,et al.  Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system , 1999, Comput. Graph..

[33]  Roberto Cipolla,et al.  A system for automatic pose-estimation from a single image in a city scene , 2002 .

[34]  Vincent Lepetit,et al.  Fully automated and stable registration for augmented reality applications , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..