W Enabling Technologies Yohan Baillot Nrl Virtual Reality Lab/itt Advanced Engineering Recent Advances in Augmented Reality

hat is augmented reality? An AR system supplements the real world with virtual (computer-generated) objects that appear to coexist in the same space as the real world. While many researchers broaden the definition of AR beyond this vision, we define an AR system to have the following properties: I combines real and virtual objects in a real environment; I runs interactively, and in real time; and I registers (aligns) real and virtual objects with each other. Note that we don't restrict this definition of AR to particular display technologies, such as a head-mounted display (HMD). Nor do we limit it to our sense of sight. AR can potentially apply to all senses, including hearing, touch, and smell. Certain AR applications also require removing real objects from the perceived environment, in addition to adding virtual objects. For example, an AR visualization of a building that stood at a certain location might remove the building that exists there today. Some researchers call the task of removing real objects mediated or diminished reality, but we consider it a subset of AR. Milgram 1 defined a continuum of real-to-virtual environments , in which AR is one part of the general area of mixed reality (Figure 1). In both augmented virtuality, in which real objects are added to virtual ones, and virtual environments (or virtual reality), the surrounding environment is virtual, while in AR the surrounding environment is real. We focus on AR and don't cover augmented virtuality or virtual environments. The beginnings of AR, as we define it, date back to Sutherland's work in the 1960s, which used a see-through HMD to present 3D graphics. 2 However, only over the past decade has there been enough work to refer to AR as a research field. In 1997, Azuma published a survey 3 that defined the field, described many problems, and summarized the developments up to that point. Since then, AR's growth and progress have been remarkable. In the late 1990s, several conferences on AR began, including the International Workshop and Symposium on Augmented Reality, the International Symposium on Mixed Reality, and the Designing Augmented Reality Environments workshop. Some well-funded organizations formed that focused on AR, notably the Mixed Reality Systems Lab in Japan and the Arvika consortium in Germany. A software toolkit (the ARToolkit) for rapidly building AR applications is now freely available at http://cause of the wealth of new developments, this field needs …

[1]  Yuichi Ohta,et al.  Share-Z: Client/Server Depth Sensing for See-Through Head-Mounted Displays , 2002, Presence: Teleoperators & Virtual Environments.

[2]  Shigeru Nakamura,et al.  A Wearable Attitude-Measurement System Using a Fiberoptic Gyroscope , 2002, Presence: Teleoperators & Virtual Environments.

[3]  Steven K. Feiner,et al.  View management for virtual and augmented reality , 2001, UIST '01.

[4]  Simon J. Julier,et al.  Authoring of physical models using mobile computers , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[5]  Ivan Poupyrev,et al.  The MagicBook - Moving Seamlessly between Reality and Virtuality , 2001, IEEE Computer Graphics and Applications.

[6]  Frank Biocca,et al.  An ultra-light and compact design and implementation of head-mounted projective displays , 2001, Proceedings IEEE Virtual Reality 2001.

[7]  Greg Welch,et al.  High-Performance Wide-Area Optical Tracking: The HiBall Tracking System , 2001, Presence: Teleoperators & Virtual Environments.

[8]  Ioannis T. Christou,et al.  Design and development issues for ARCHEOGUIDE: An Augmented Reality based cultural heritage on-site guide , 2001 .

[9]  Jannick P. Rolland,et al.  A Survey of Tracking Technologies for Virtual Environments , 2001 .

[10]  Ryugo Kijima,et al.  A Development of Reflex HMD - HMD with time delay compensation capability , 2001 .

[11]  Blair MacIntyre,et al.  Ghosts in the Machine: Integrating 2D Video Actors into a 3D AR System , 2001 .

[12]  Steve Mann,et al.  Telepointer: Hands-free completely self-contained wearable visual augmented reality without headwear and without any infrastructural reliance , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[13]  Bruce H. Thomas,et al.  ARQuake: an outdoor/indoor augmented reality first person application , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[14]  Hiroaki Ueda,et al.  A forgettable near eye display , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[15]  Yongduek Seo,et al.  Weakly calibrated video-based augmented reality: embedding and rendering through virtual camera , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[16]  A. Takagi,et al.  Development of a stereo video see-through HMD for AR systems , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[17]  Yakup Genc,et al.  Augmented workspace: designing an AR testbed , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[18]  Volker Coors,et al.  Matching buildings: pose estimation in an urban environment , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[19]  Marius S. Vassiliou,et al.  A wearable augmented reality testbed for navigation and control, built solely with commercial-off-the-shelf (COTS) hardware , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[20]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[21]  Blair MacIntyre,et al.  Adapting to dynamic registration errors using level of error (LOE) filtering , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[22]  Ivan Poupyrev,et al.  Virtual object manipulation on a table-top AR environment , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[23]  Vincent Lepetit,et al.  Handling occlusion in augmented reality systems: a semi-automatic method , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[24]  Ronald Azuma,et al.  Autocalibration of an electronic compass in an outdoor augmented reality system , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[25]  Steven K. Feiner,et al.  Information filtering for mobile augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[26]  Suya You,et al.  Camera tracking for augmented reality media , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[27]  Laurent Vaissie,et al.  Accuracy of rendered depth in head-mounted displays: role of eyepoint location , 2000, Defense, Security, and Sensing.

[28]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[29]  Tsuneo Yoshikawa,et al.  Accurate image overlay on video see-through HMDs using vision and accelerometers , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[30]  Masahiko Inami,et al.  Visuo-haptic display using head-mounted projector , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[31]  Ronald Azuma,et al.  Tracking in unprepared environments for augmented reality systems , 1999, Comput. Graph..

[32]  Michael Gervautz,et al.  Occlusion in collaborative augmented environments , 1999, Comput. Graph..

[33]  Steven K. Feiner,et al.  Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system , 1999, Comput. Graph..

[34]  Adam L. Janin,et al.  Several devils in the details: making an AR application work in the airplane factory , 1999 .

[35]  Ulrich Neumann,et al.  A multi-ring fiducial system and an intensity-invariant detection method for scalable augmented reality , 1999 .

[36]  Yasuhiro Mukaigawa,et al.  Photometric image-based rendering for virtual lighting image synthesis , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[37]  Nassir Navab,et al.  Scene augmentation via the fusion of industrial drawings and uncalibrated images with a view to marker-less calibration , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[38]  Bruce H. Thomas,et al.  Integrating virtual and augmented realities in an outdoor application , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[39]  Greg Welch,et al.  Table-top spatially-augmented realty: bringing physical models to life with projected imagery , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[40]  Nassir Navab,et al.  Merging visible and invisible: two Camera-Augmented Mobile C-arm (CAMC) applications , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[41]  Steven K. Feiner,et al.  Enveloping users and computers in a collaborative 3D augmented reality , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[42]  Tyrone L. Vincent,et al.  An adaptive estimator for registration in augmented reality , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[43]  Gavin S. P. Miller,et al.  A real-time low-latency hardware light-field renderer , 1999, SIGGRAPH.

[44]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[45]  Reinhold Behringer,et al.  Registration for outdoor augmented reality applications using computer vision techniques and hybrid sensors , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[46]  Simon J. Julier,et al.  The software architecture of a real-time battlefield visualization virtual environment , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[47]  Frederick P. Brooks,et al.  What's Real About Virtual Reality? , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[48]  Ronald Azuma,et al.  A motion-stabilized outdoor augmented reality system , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[49]  Suya You,et al.  Natural Feature Tracking for Augmented Reality , 1999, IEEE Trans. Multim..

[50]  Takeo Kanade,et al.  Virtualized Reality : Digitizing a 3D Time-Varying Event As Is and in Real Time , 1999 .

[51]  Hirokazu Kato,et al.  Collaborative Mixed Reality , 1999 .

[52]  J. Urgen Stauder Augmented Reality with Automatic Illumination Control Incorporating Ellipsoidal Models , 1999 .

[53]  Jun Rekimoto,et al.  Augment-able reality: situated communication through physical and digital spaces , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[54]  Ramesh Raskar,et al.  Augmented Reality Visualization for Laparoscopic Surgery , 1998, MICCAI.

[55]  Erik Viirre,et al.  The Virtual Retinal Display: A new Display Technology using Scanned Laser Light , 1998 .

[56]  Michael Harrington,et al.  Constellation: a wide-range wireless motion-tracking system for augmented reality and virtual set applications , 1998, SIGGRAPH.

[57]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[58]  Frank Biocca,et al.  Virtual Eyes Can Rearrange Your Body: Adaptation to Visual Displacement in See-Through, Head-Mounted Displays , 1998, Presence.

[59]  George A. Bekey,et al.  Compensation for end to end delays in a VR system , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[60]  Ulrich Neumann,et al.  Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[61]  Hideyuki Tamura,et al.  AR/sup 2/Hockey: a case study of collaborative augmented reality , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[62]  Hiroshi Ishii,et al.  Illuminating light: an optical design tool with a luminous-tangible interface , 1998, CHI.

[63]  Kiriakos N. Kutulakos,et al.  Calibration-Free Augmented Reality , 1998, IEEE Trans. Vis. Comput. Graph..

[64]  Noa M. Rensing,et al.  Eyeglass-based systems for wearable computing , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[65]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[66]  Alex Pentland,et al.  Stochasticks: augmenting the billiards experience with probabilistic vision and wearable computers , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[67]  Michael Gervautz,et al.  The Personal Interaction Panel – a Two‐Handed Interface for Augmented Reality , 1997, Comput. Graph. Forum.

[68]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[69]  Jun Rekimoto,et al.  NaviCam:A Magnifying Glass Approach to Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[70]  Suzanne Weghorst,et al.  Augmented reality and Parkinson's disease , 1997, CACM.

[71]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[72]  Richard L. Holloway,et al.  Registration Error Analysis for Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[73]  Leonard McMillan,et al.  Post-rendering 3D warping , 1997, SI3D.

[74]  Mark A. Livingston,et al.  Managing latency in complex augmented reality systems , 1997, SI3D.

[75]  Bernard D. Adelstein,et al.  Factors influencing operator interaction with virtual objects viewed via head-mounted see-through displays: viewing conditions and rendering latency , 1997, Proceedings of IEEE 1997 Annual International Symposium on Virtual Reality.

[76]  Rick Cavallaro,et al.  The FoxTrax Hockey Puck Tracking System , 1997, IEEE Computer Graphics and Applications.

[77]  Steve Mann,et al.  Wearable Computing: A First Step Toward Personal Imaging , 1997, Computer.

[78]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[79]  Paul Milgram,et al.  Perceptual issues in augmented reality , 1996, Electronic Imaging.

[80]  J. Rekimoto,et al.  Transvision: a hand-held augmented reality system for collaborative design , 1996 .

[81]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[82]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.