A Survey of Augmented Reality Technologies, Applications and Limitations

We are on the verge of ubiquitously adopting Augmented Reality (AR) technologies to enhance our percep- tion and help us see, hear, and feel our environments in new and enriched ways. AR will support us in fields such as education, maintenance, design and reconnaissance, to name but a few. This paper describes the field of AR, including a brief definition and development history, the enabling technologies and their characteristics. It surveys the state of the art by reviewing some recent applications of AR technology as well as some known limitations regarding human factors in the use of AR systems that developers will need to overcome.

[1]  Alois Ferscha,et al.  Pervasive information acquisition for mobile AR-navigation systems , 2003, 2003 Proceedings Fifth IEEE Workshop on Mobile Computing Systems and Applications.

[2]  Bjorn Ekengren,et al.  Mobile Augmented Reality , 2010 .

[3]  Randy Pausch,et al.  Virtual reality on a WIM: interactive worlds in miniature , 1995, CHI '95.

[4]  Qunsheng Peng,et al.  Light source estimation of outdoor scenes for mixed reality , 2009, The Visual Computer.

[5]  Mark Billinghurst,et al.  Face to face collaborative AR on mobile phones , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[6]  Ava Fatah gen. Schieck,et al.  ARTHUR: A Collaborative Augmented Environment for Architectural Design and Urban Planning , 2004, J. Virtual Real. Broadcast..

[7]  Steven K. Feiner,et al.  Enveloping users and computers in a collaborative 3D augmented reality , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[8]  J. C. R. Licklider,et al.  Man-Computer Symbiosis , 1960 .

[9]  Alex Pentland,et al.  Stochasticks: augmenting the billiards experience with probabilistic vision and wearable computers , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[10]  Frank Biocca,et al.  Virtual Eyes Can Rearrange Your Body: Adaptation to Visual Displacement in See-Through, Head-Mounted Displays , 1998, Presence.

[11]  Marius S. Vassiliou,et al.  A wearable augmented reality testbed for navigation and control, built solely with commercial-off-the-shelf (COTS) hardware , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[12]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[13]  F. Raab,et al.  Magnetic Position and Orientation Tracking System , 1979, IEEE Transactions on Aerospace and Electronic Systems.

[14]  Jurjen Caarls,et al.  Augmented Reality for Art, Design and Cultural Heritage—System Design and Evaluation , 2009, EURASIP J. Image Video Process..

[15]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[16]  Hideyuki Tamura,et al.  AR/sup 2/Hockey: a case study of collaborative augmented reality , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[17]  Ronald Azuma,et al.  The Challenge of Making Augmented Reality Work Outdoors , 1999 .

[18]  Ramesh Raskar,et al.  Modern approaches to augmented reality , 2005, SIGGRAPH Courses.

[19]  Bruce H. Thomas,et al.  Integrating virtual and augmented realities in an outdoor application , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[20]  Dieter Schmalstieg,et al.  First steps towards handheld augmented reality , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[21]  Didier Stricker,et al.  Archeoguide: first results of an augmented reality, mobile computing system in cultural heritage sites , 2001, VAST '01.

[22]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[23]  Hiroaki Ueda,et al.  A forgettable near eye display , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[24]  Paul A. Beardsley,et al.  Natural video matting using camera arrays , 2006, ACM Trans. Graph..

[25]  Laurent Vaissie,et al.  Accuracy of rendered depth in head-mounted displays: role of eyepoint location , 2000, Defense, Security, and Sensing.

[26]  J. Rekimoto,et al.  Transvision: a hand-held augmented reality system for collaborative design , 1996 .

[27]  David G. Lowe,et al.  What and Where: 3D Object Recognition with Accurate Pose , 2006, Toward Category-Level Object Recognition.

[28]  Ivan Poupyrev,et al.  The MagicBook - Moving Seamlessly between Reality and Virtuality , 2001, IEEE Computer Graphics and Applications.

[29]  Éric Marchand,et al.  A real-time tracker for markerless augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[30]  Bernard D. Adelstein,et al.  Factors influencing operator interaction with virtual objects viewed via head-mounted see-through displays: viewing conditions and rendering latency , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[31]  Alois Ferscha,et al.  Augmented reality navigation systems , 2006, Universal Access in the Information Society.

[32]  Angela Chang,et al.  Audio-haptic feedback in mobile phones , 2005, CHI Extended Abstracts.

[33]  Michitaka Hirose,et al.  Invisible Interface for the Immersive Virtual World , 2001, EGVE/IPT.

[34]  Petr Musílek,et al.  A keystroke and pointer control input interface for wearable computers , 2006, Fourth Annual IEEE International Conference on Pervasive Computing and Communications (PERCOM'06).

[35]  Raffaele de Amicis,et al.  Spacedesign: A Mixed Reality Workspace for Aesthetic Industrial Design , 2002, ISMAR.

[36]  B. A. MacDonald,et al.  Developer Oriented Visualisation of a Robot Program An Augmented Reality Approach , 2006 .

[37]  Gregory D. Abowd,et al.  Charting past, present, and future research in ubiquitous computing , 2000, TCHI.

[38]  Desney S. Tan,et al.  Tiles: A Mixed Reality Authoring Interface , 2001, INTERACT.

[39]  Simon J. Julier,et al.  The software architecture of a real-time battlefield visualization virtual environment , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[40]  Charles E. Hughes,et al.  The Evolution of a Framework for Mixed Reality Experiences , 2007 .

[41]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[42]  George D. Stetten,et al.  Real time tomographic reflection: phantoms for calibration and biopsy , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[43]  Adam L. Janin,et al.  Several devils in the details: making an AR application work in the airplane factory , 1999 .

[44]  Klaus Bengler,et al.  Augmented Reality in Cars Requirements and Constraints Invited Speech , 2006 .

[45]  Hirokazu Kato,et al.  Multi-user mixed reality system ‘Gulliver’s World’: a case study on collaborative edutainment at the intersection of material and virtual worlds , 2006, Virtual Reality.

[46]  Abderrahmane Kheddar,et al.  Tactile interfaces: a state-of-the-art survey , 2004 .

[47]  Christian Sandor,et al.  A rapid prototyping software infrastructure for user interfaces in ubiquitous augmented reality , 2005, Personal and Ubiquitous Computing.

[48]  Jun Rekimoto,et al.  iCam: Precise at-a-Distance Interaction in the Physical Environment , 2006, Pervasive.

[49]  J. Mcneff.  The global positioning system , 2002 .

[50]  Martin White,et al.  Web3D and Augmented Reality to support engineering education , 2004 .

[51]  Frank Biocca,et al.  Development of Head-Mounted Projection Displays for Distributed, Collaborative, Augmented Reality Applications , 2005, Presence: Teleoperators & Virtual Environments.

[52]  Ronald Azuma,et al.  Performance analysis of an outdoor augmented reality tracking system that relies upon a few mobile beacons , 2006, 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.

[53]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[54]  A. Takagi,et al.  Development of a stereo video see-through HMD for AR systems , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[55]  Benjamin Lok,et al.  Virtual Human + Tangible Interface = Mixed Reality Human An Initial Exploration with a Virtual Breast Exam Patient , 2008, 2008 IEEE Virtual Reality Conference.

[56]  Woodrow Barfield,et al.  Fundamentals of Wearable Computers and Augumented Reality , 2000 .

[57]  Hirokazu Kato,et al.  Collaborative Mixed Reality , 1999 .

[58]  Jani Mäntyjärvi,et al.  Accelerometer-based gesture control for a design environment , 2006, Personal and Ubiquitous Computing.

[59]  T. Sugihara,et al.  32.4: A Lightweight 3‐D HMD with Accommodative Compensation , 1998 .

[60]  Andreas Kunz,et al.  blue-c: a spatially immersive display and 3D video portal for telepresence , 2003, ACM Trans. Graph..

[61]  Vincent Hayward,et al.  Haptic interfaces and devices , 2004 .

[62]  Michael Herczeg,et al.  Collaborative and Constructive Learning of Elementary School Children in Experiental Learning Spaces along the Virtuality Continuum , 2002, MuC.

[63]  Hideyuki Tamura,et al.  Mixed Reality: Future Dreams Seen at the Border between Real and Virtual Worlds , 2001, IEEE Computer Graphics and Applications.

[64]  Woodrow Barfield,et al.  Boeing’s Wire Bundle Assembly Project , 2001 .

[65]  Jannick P. Rolland,et al.  Optical versus Video See-Through Head-Mounted Displays , 2001 .

[66]  Elliot Soloway,et al.  The next generation in human-computer interaction , 1996, CACM.

[67]  Hideyuki Tamura.  Steady Steps and Giant Leap Toward Practical Mixed Reality Systems and Applications , 2002 .

[68]  Steven K. Feiner,et al.  View management for virtual and augmented reality , 2001, UIST '01.

[69]  James H. Aylor,et al.  Computer for the 21st Century , 1999, Computer.

[70]  H. Ritter,et al.  Interactive online learning , 2007, Pattern Recognition and Image Analysis.

[71]  Nassir Navab,et al.  Merging visible and invisible: two Camera-Augmented Mobile C-arm (CAMC) applications , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[72]  Bruce H. Thomas,et al.  Tinmith-Metro: new outdoor techniques for creating city models with an augmented reality wearable computer , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[73]  Christian Sandor,et al.  Experimental evaluation of an augmented reality visualization for directing a car driver's attention , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[74]  Armin Hammand,et al.  Telegeoinformatics: Location-based Computing and Services , 2004 .

[75]  Adrian David Cheok,et al.  Online 6 DOF augmented reality registration from natural features , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[76]  Steven K. Feiner,et al.  Situated documentaries: embedding multimedia presentations in the real world , 1999, Digest of Papers. Third International Symposium on Wearable Computers.

[77]  Eric Foxlin,et al.  Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[78]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[79]  Henry Been-Lirn Duh,et al.  Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[80]  Michael Harrington,et al.  WearTrack: a self-referenced head and hand tracker for wearable computers and portable VR , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[81]  Paramvir Bahl,et al.  RADAR: an in-building RF-based user location and tracking system , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[82]  Jun Rekimoto,et al.  NaviCam:A Magnifying Glass Approach to Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[83]  Simon J. Julier,et al.  Tracking: how hard can it be? , 2002, IEEE Computer Graphics and Applications.

[84]  Peter Antoniac,et al.  Marisil – Mobile User Interface Framework for Virtual Enterprise , 2001 .

[85]  George Papagiannakis,et al.  Mixing virtual and real scenes in the site of ancient Pompeii , 2005, Comput. Animat. Virtual Worlds.

[86]  Wolfgang Friedrich,et al.  ARVIKA-augmented reality for development, production and service , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[87]  Luc Van Gool,et al.  Blue-c: a spatially immersive display and 3D video portal for telepresence , 2003, IPT/EGVE.

[88]  M. Turcsányi-Szabó,et al.  Augmented Reality in Education , 2018, Handbook of Mobile Teaching and Learning.

[89]  J. Rolland,et al.  Head-worn displays: a review , 2006, Journal of Display Technology.

[90]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[91]  Rick Cavallaro,et al.  The FoxTrax Hockey Puck Tracking System , 1997, IEEE Computer Graphics and Applications.

[92]  JulierSimon,et al.  Information Filtering for Mobile Augmented Reality , 2002 .

[93]  Hirokazu Kato,et al.  Game-City: a ubiquitous large area multi-interface mixed reality game space for wearable computers , 2002, Proceedings. Sixth International Symposium on Wearable Computers,.

[94]  Frank Sauer,et al.  Reality Augmentation for Medical Procedures: System Architecture, Single Camera Marker Tracking, and System Evaluation , 2006, International Journal of Computer Vision.

[95]  Steven K. Feiner,et al.  Information filtering for mobile augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[96]  Paul Milgram,et al.  Perceptual issues in augmented reality , 1996, Electronic Imaging.

[97]  T. P. Caudell,et al.  Augmented reality: an application of heads-up display technology to manual manufacturing processes , 1992, Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences.

[98]  Nuno Correia,et al.  Storing user experiences in mixed reality using hypermedia , 2006, The Visual Computer.

[99]  Takeshi Naemura,et al.  Laser-plasma scanning 3D display for putting digital contents in free space , 2008, Electronic Imaging.

[100]  David W. Murray,et al.  Wearable hand activity recognition for event summarization , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[101]  Steve Mann,et al.  Wearable Computing: A First Step Toward Personal Imaging , 1997, Computer.

[102]  Julien Pilet,et al.  Augmented reality for non-rigid surfaces , 2008 .

[103]  Greg Welch,et al.  Table-top spatially-augmented realty: bringing physical models to life with projected imagery , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[104]  Anind K. Dey,et al.  Simulated augmented reality windshield display as a cognitive mapping aid for elder driver navigation , 2009, CHI.

[105]  Luc Van Gool,et al.  Markerless augmented reality with a real-time affine region tracker , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[106]  Steve Benford,et al.  Orchestrating a mixed reality game 'on the ground' , 2004, CHI.

[107]  Hannes Kaufmann.  Construct3D: an augmented reality application for mathematics and geometry education , 2002, MULTIMEDIA '02.

[108]  Steven Feiner.  Wearing It Out : First Steps Toward Mobile Augmented Reality Systems , 1999 .

[109]  Mark Billinghurst,et al.  An occlusion capable optical see-through head mount display for supporting co-located collaboration , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[110]  Steve Benford,et al.  Understanding and constructing shared spaces with mixed-reality boundaries , 1998, TCHI.

[111]  Charles E. Hughes,et al.  Mixed reality in education, entertainment, and training , 2005, IEEE Computer Graphics and Applications.

[112]  Simon J. Julier,et al.  Authoring of physical models using mobile computers , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[113]  Tobias Höllerer,et al.  Hybrid Feature Tracking and User Interaction for Markerless Augmented Reality , 2008, 2008 IEEE Virtual Reality Conference.

[114]  Andy Cockburn,et al.  FingARtips: gesture based direct manipulation in Augmented Reality , 2004, GRAPHITE '04.

[115]  Brian T. Schowengerdt,et al.  Binocular retinal scanning laser display with integrated focus cues for ocular accommodation , 2003, IS&T/SPIE Electronic Imaging.

[116]  Ted Kremenek,et al.  A Probabilistic Room Location Service for Wireless Networked Environments , 2001, UbiComp.

[117]  Jae Yeol Lee,et al.  Visualization and interaction of pervasive services using context-aware augmented reality , 2008, Expert Syst. Appl..

[118]  Dieter Schmalstieg,et al.  “Studierstube”: An environment for collaboration in augmented reality , 1998, Virtual Reality.

[119]  Manit Arya,et al.  THREE‐DIMENSIONAL VISUALIZATION IN LAPAROSCOPIC SURGERY , 2006, BJU international.

[120]  Wayne Piekarski,et al.  ARQuake: the outdoor augmented reality gaming system , 2002, CACM.

[121]  Ruzena Bajcsy,et al.  Immersive 3D Environment for Remote Collaboration and Training of Physical Activities , 2008, 2008 IEEE Virtual Reality Conference.

[122]  Roberta L. Klatzky,et al.  Personal guidance system for the visually impaired , 1994, ASSETS.

[123]  Pattie Maes,et al.  WUW - wear Ur world: a wearable gestural interface , 2009, CHI Extended Abstracts.

[124]  Erik Brunvand,et al.  Fast ray tracing and the potential effects on graphics and gaming courses , 2008, Comput. Graph..

[125]  Dieter Schmalstieg,et al.  Construct3D: A Virtual Reality Application for Mathematics and Geometry Education , 2000, Education and Information Technologies.

[126]  Rosalind W. Picard.  Affective computing: challenges , 2003, Int. J. Hum. Comput. Stud..

[127]  Rafael Radkowski,et al.  AR-bowling: immersive and realistic game play in real environments using augmented reality , 2004, ACE '04.

[128]  Frank Biocca,et al.  Comparative effectiveness of augmented reality in object assembly , 2003, CHI '03.

[129]  Steven K. Feiner,et al.  Augmented reality: a new way of seeing. , 2002, Scientific American.

[130]  William A. Mackaness,et al.  Development of a Speech-Based Augmented Reality System to Support Exploration of Cityscape , 2006, Trans. GIS.

[131]  Hiroshi Ishii,et al.  Tangible bits: towards seamless interfaces between people, bits and atoms , 1997, CHI.

[132]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[133]  Dieter Schmalstieg,et al.  Bridging multiple user interface dimensions with augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[134]  Steven K. Feiner,et al.  Unit: modular development of distributed interaction techniques for highly interactive user interfaces , 2004, GRAPHITE '04.

[135]  Terry Winograd,et al.  GUIDe: gaze-enhanced UI design , 2007, CHI Extended Abstracts.

[136]  Oliver Bimber,et al.  Video see-through AR on consumer cell-phones , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[137]  Phillip S. Dunston,et al.  Identification of application areas for Augmented Reality in industrial construction based on technology suitability , 2008 .

[138]  James Church,et al.  Wearable sensor badge and sensor jacket for context awareness , 1999, Digest of Papers. Third International Symposium on Wearable Computers.

[139]  Fabian Doil,et al.  Augmented Reality-based factory planning - an application tailored to industrial needs , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[140]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[141]  R. Sader,et al.  Development of an Augmented Reality System for intraoperative navigation in maxillofacial surgery , 2004 .

[142]  Ramesh Raskar,et al.  Session details: Course 17: Spatial augmented reality: merging real and virtual worlds , 2007, ACM SIGGRAPH 2007 courses.

[143]  Didier Stricker,et al.  Augmented Reality for Exterior Construction Applications , 2001 .

[144]  Nitaigour-Premchand Mahalik,et al.  A Reliable New 2-Stage Distributed Interactive TGS System Based on GIS Database and Augmented Reality , 2006, IEICE Trans. Inf. Syst..

[145]  Jiejie Zhu,et al.  Virtual Reality and Mixed Reality for Virtual Learning Environments , 2006, Edutainment.