Advanced Visual and Instruction Systems for Maintenance Support (AVIS-MS)

Abstract : Complexity, customization, and packaging of military platforms and systems increase maintenance difficulty at the same time as the available pool of skilled technical personnel may be shrinking. In this environment maintenance training, technical order presentation, and flight-line operational practice may need to adopt "just-in-time" procedural aids. Moreover, the realities of real-world maintenance may not permit the hardware indulgences and rigid controls of laboratory settings for visualization and training systems, and at the same time the actual activities of maintainers will challenge requirements for portable or wearable devices. This project has investigated technologies that may be used by Air Force maintainers for training or job aids.

[1]  Greg Welch,et al.  SCAAT: incremental tracking with incomplete information , 1997, SIGGRAPH.

[2]  I. Bartenieff,et al.  Body Movement: Coping with the Environment , 1980 .

[3]  Jun Rekimoto,et al.  Augmented surfaces: a spatially continuous work space for hybrid computing environments , 1999, CHI '99.

[4]  Jeffrey Mark Siskind,et al.  Grounding the Lexical Semantics of Verbs in Visual Perception using Force Dynamics and Event Logic , 1999, J. Artif. Intell. Res..

[5]  Jannick P. Rolland,et al.  A compact optical see-through head-worn display with occlusion support , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[6]  Ioannis T. Christou,et al.  Design and development issues for ARCHEOGUIDE: An Augmented Reality based cultural heritage on-site guide , 2001 .

[7]  Michael Gleicher,et al.  Comparing Constraint-Based Motion Editing Methods , 2001, Graph. Model..

[8]  Mark Billinghurst,et al.  Designing augmented reality interfaces , 2005, COMG.

[9]  Holger Regenbrecht,et al.  Augmented Reality Projects in Automotive and Aerospace Industry , 2005 .

[10]  Ronald Azuma,et al.  A motion-stabilized outdoor augmented reality system , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[11]  Steven K. Feiner,et al.  Information filtering for mobile augmented reality , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[12]  Steven K. Feiner,et al.  View management for virtual and augmented reality , 2001, UIST '01.

[13]  Steven K. Feiner,et al.  Exploring MARS: developing indoor and outdoor user interfaces to a mobile augmented reality system , 1999, Comput. Graph..

[14]  Eric Foxlin,et al.  Circular data matrix fiducial system and robust image processing for a wearable vision-inertial self-tracker , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[15]  Ronald Azuma,et al.  Tracking in unprepared environments for augmented reality systems , 1999, Comput. Graph..

[16]  Bruce H. Thomas,et al.  ARQuake: an outdoor/indoor augmented reality first person application , 2000, Digest of Papers. Fourth International Symposium on Wearable Computers.

[17]  Anselmo Lastra,et al.  Life-sized projector-based dioramas , 2001, VRST '01.

[18]  Hirokazu Kato,et al.  Collaborative Mixed Reality , 1999 .

[19]  Steven K. Feiner,et al.  Enveloping users and computers in a collaborative 3D augmented reality , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[20]  Michitaka Hirose,et al.  Projected augmentation II - a scalable architecture for multi projector based AR-systems based on 'Projected Applications' , 2005, Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'05).

[21]  Norman I. Badler,et al.  Motion Abstraction and Mapping with Spatial Constraints , 1998, CAPTECH.

[22]  Norman I. Badler,et al.  The EMOTE model for effort and shape , 2000, SIGGRAPH.

[23]  Jennifer Healey,et al.  Augmented Reality through Wearable Computing , 1997, Presence: Teleoperators & Virtual Environments.

[24]  Hideyuki Tamura,et al.  AR/sup 2/Hockey: a case study of collaborative augmented reality , 1998, Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No.98CB36180).

[25]  Paul A. Beardsley,et al.  RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors , 2004, ACM Trans. Graph..

[26]  Eric Foxlin,et al.  Miniaturization, calibration & accuracy evaluation of a hybrid self-tracker , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[27]  Nassir Navab,et al.  Industrial augmented reality (IAR): challenges in design and commercialization of killer apps , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[28]  Simon J. Julier,et al.  The software architecture of a real-time battlefield visualization virtual environment , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[29]  Andrew W. Fitzgibbon,et al.  Markerless tracking using planar structures in the scene , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[30]  Frank Biocca,et al.  An ultra-light and compact design and implementation of head-mounted projective displays , 2001, Proceedings IEEE Virtual Reality 2001.

[31]  Steven K. Feiner,et al.  Augmented Reality in Architectural Construction, Inspection, and Renovation , 1996 .

[32]  Greg Welch,et al.  Table-top spatially-augmented realty: bringing physical models to life with projected imagery , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[33]  Norman I. Badler,et al.  Instructions, Intentions and Expectations , 1995, Artif. Intell..

[34]  J. Urgen Stauder.  Augmented Reality with Automatic Illumination Control Incorporating Ellipsoidal Models , 1999 .

[35]  Paul E. Debevec,et al.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 1998, SIGGRAPH '08.

[36]  Ronald Azuma,et al.  Tracking requirements for augmented reality , 1993, CACM.

[37]  Blandine de Laval,et al.  STARMATE : Using Augmented Reality technology for computer guided maintenance of complex mechanical elements , 2002 .

[38]  Thomas Reicher,et al.  A framework for dynamically adaptable augmented reality systems , 2004 .

[39]  Mark Steedman,et al.  Animated conversation: rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents , 1994, SIGGRAPH.

[40]  Michael Harrington,et al.  Constellation: a wide-range wireless motion-tracking system for augmented reality and virtual set applications , 1998, SIGGRAPH.

[41]  James L. Crowley,et al.  Finger Tracking as an Input Device for Augmented Reality , 1995 .

[42]  Ronald Azuma,et al.  Hybrid inertial and vision tracking for augmented reality registration , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[43]  J. Figue,et al.  A generic model for Immersive Documentation Environment and applications , 1998 .

[44]  Bruce H. Thomas,et al.  Integrating virtual and augmented realities in an outdoor application , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[45]  Michael Gervautz,et al.  Occlusion in collaborative augmented environments , 1999, Comput. Graph..

[46]  Mark A. Livingston,et al.  Superior augmented reality registration by integrating landmark tracking and magnetic tracking , 1996, SIGGRAPH.

[47]  Ronald Azuma,et al.  Autocalibration of an electronic compass in an outdoor augmented reality system , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[48]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[49]  Hiroyuki Ohno,et al.  An optical see-through display for mutual occlusion with a real-time stereovision system , 2001, Comput. Graph..

[50]  Norman I. Badler,et al.  ACUMEN: amplifying control and understanding of multiple entities , 2002, AAMAS '02.

[51]  Alex Pentland,et al.  Stochasticks: augmenting the billiards experience with probabilistic vision and wearable computers , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[52]  Yasuhiro Mukaigawa,et al.  Photometric image-based rendering for virtual lighting image synthesis , 1999, Proceedings 2nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99).

[53]  Gordon Wetzstein,et al.  Enabling view-dependent stereoscopic projection in real environments , 2005, SIGGRAPH '05.

[54]  Benjamin Lok.  Toward the merging of real and virtual spaces , 2004, CACM.

[55]  Norman I. Badler,et al.  Animation control for real-time virtual humans , 1999, CACM.

[56]  Steven K. Feiner,et al.  Knowledge-based augmented reality , 1993, CACM.

[57]  Erik Viirre,et al.  The Virtual Retinal Display: A new Display Technology using Scanned Laser Light , 1998 .

[58]  Blair MacIntyre,et al.  DART: a toolkit for rapid design exploration of augmented reality experiences , 2005, ACM Trans. Graph..

[59]  Rajeev Sharma,et al.  AREAS: augmented reality for evaluating assembly sequences , 1999 .

[60]  Pieter Abbeel,et al.  Apprenticeship learning via inverse reinforcement learning , 2004, ICML.

[61]  Daphna Buchsbaum,et al.  Imitation as a first step to social learning in synthetic characters: a graph-based approach , 2005, SCA '05.

[62]  Éric Marchand,et al.  A real-time tracker for markerless augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[63]  Paul A. Beardsley,et al.  Photosensing wireless tags for geometric procedures , 2005, CACM.

[64]  Michael Gervautz,et al.  The Personal Interaction Panel – a Two‐Handed Interface for Augmented Reality , 1997, Comput. Graph. Forum.

[65]  Ronald Azuma,et al.  A survey of augmented reality" Presence: Teleoperators and virtual environments , 1997 .

[66]  Ulrich Neumann,et al.  Dynamic registration correction in video-based augmented reality systems , 1995, IEEE Computer Graphics and Applications.

[67]  Greg Welch,et al.  High-Performance Wide-Area Optical Tracking: The HiBall Tracking System , 2001, Presence: Teleoperators & Virtual Environments.

[68]  J. Rekimoto,et al.  Transvision: a hand-held augmented reality system for collaborative design , 1996 .

[69]  Ivan Poupyrev,et al.  The MagicBook - Moving Seamlessly between Reality and Virtuality , 2001, IEEE Computer Graphics and Applications.

[70]  Suya You,et al.  Camera tracking for augmented reality media , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[71]  Kiriakos N. Kutulakos,et al.  Calibration-Free Augmented Reality , 1998, IEEE Trans. Vis. Comput. Graph..

[72]  Ivan Poupyrev,et al.  Virtual object manipulation on a table-top AR environment , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[73]  Norman I. Badler,et al.  Making Them Move: Mechanics, Control & Animation of Articulated Figures , 1990 .

[74]  Paul Lukowicz,et al.  Fin-germouse: A wearable hand tracking system , 2002 .

[75]  Henry Sowizral,et al.  Note on dynamics of human head motions and on predictive filtering of head-set orientations , 1995, Other Conferences.

[76]  Jun Rekimoto,et al.  NaviCam:A Magnifying Glass Approach to Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[77]  Soh-Khim Ong,et al.  The virtual interaction panel: an easy control tool in augmented reality systems , 2004, Comput. Animat. Virtual Worlds.

[78]  Nassir Navab,et al.  Automated initialization for marker-less tracking: a sensor fusion approach , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[79]  Jun Rekimoto,et al.  Augment-able reality: situated communication through physical and digital spaces , 1998, Digest of Papers. Second International Symposium on Wearable Computers (Cat. No.98EX215).

[80]  Tsuneo Yoshikawa,et al.  Accurate image overlay on video see-through HMDs using vision and accelerometers , 2000, Proceedings IEEE Virtual Reality 2000 (Cat. No.00CB37048).

[81]  Yakup Genc,et al.  Augmented workspace: designing an AR testbed , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[82]  Simon J. Julier,et al.  Authoring of physical models using mobile computers , 2001, Proceedings Fifth International Symposium on Wearable Computers.

[83]  Tom Drummond,et al.  Sensor fusion and occlusion refinement for tablet-based AR , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[84]  Norman I. Badler,et al.  A Parameterized Action Representation for Virtual Human Agents , 1998 .

[85]  Hiroyuki Yamamoto,et al.  A head tracking method using bird's-eye view camera and gyroscope , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[86]  Reinhold Behringer,et al.  Registration for outdoor augmented reality applications using computer vision techniques and hybrid sensors , 1999, Proceedings IEEE Virtual Reality (Cat. No. 99CB36316).

[87]  Dieter Schmalstieg,et al.  “Studierstube”: An environment for collaboration in augmented reality , 1998, Virtual Reality.

[88]  G. A. Giraldi,et al.  Introduction to Augmented Reality , 2003 .

[89]  Eric Foxlin,et al.  VIS-Tracker: a wearable vision-inertial self-tracker , 2003, IEEE Virtual Reality, 2003. Proceedings..

[90]  Frank J. Ferrin.  Survey of helmet tracking technologies , 1991, Medical Imaging.

[91]  Steven K. Feiner,et al.  A touring machine: Prototyping 3D mobile augmented reality systems for exploring the urban environment , 1997, Digest of Papers. First International Symposium on Wearable Computers.

[92]  Bernd Schwald,et al.  An Augmented Reality System for Training and Assistence to Maintenance in the Industrial Context , 2003, WSCG.

[93]  Markus H. Gross,et al.  Embedding imperceptible patterns into projected images for simultaneous acquisition and display , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[94]  Vincent Lepetit,et al.  Handling occlusion in augmented reality systems: a semi-automatic method , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[95]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[96]  Volker Coors,et al.  Matching buildings: pose estimation in an urban environment , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[97]  Stefan Schaal,et al.  Robot Learning From Demonstration , 1997, ICML.

[98]  Frank Biocca,et al.  A Survey of Position Trackers , 1992, Presence: Teleoperators & Virtual Environments.

[99]  Norman I. Badler,et al.  Acquiring and validating motion qualities from live limb gestures , 2005, Graph. Model..

[100]  Gabriel Zachmann,et al.  Distortion correction of magnetic fields for position tracking , 1997, Proceedings Computer Graphics International.

[101]  Jannick P. Rolland,et al.  A Survey of Tracking Technologies for Virtual Environments , 2001 .

[102]  Didier Stricker,et al.  Augmented reality for construction tasks: doorlock assembly , 1999 .

[103]  Hiroyuki Ohno,et al.  An optical see-through display for mutual occlusion of real and virtual environments , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[104]  Yongduek Seo,et al.  Weakly calibrated video-based augmented reality: embedding and rendering through virtual camera , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[105]  Gerard Jounghyun Kim,et al.  Immersive authoring of tangible augmented reality applications , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[106]  W. Brent Seales,et al.  Multi-projector displays using camera-based registration , 1999, Proceedings Visualization '99 (Cat. No.99CB37067).

[107]  Dieter Schmalstieg,et al.  Ubiquitous tracking for augmented reality , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[108]  Ulrich Neumann,et al.  A multi-ring fiducial system and an intensity-invariant detection method for scalable augmented reality , 1999 .

[109]  Soh-Khim Ong,et al.  Registration based on projective reconstruction technique for augmented reality systems , 2005, IEEE Transactions on Visualization and Computer Graphics.

[110]  Shigeru Nakamura,et al.  A Wearable Attitude-Measurement System Using a Fiberoptic Gyroscope , 2002, Presence: Teleoperators & Virtual Environments.

[111]  Suya You,et al.  Natural Feature Tracking for Augmented Reality , 1999, IEEE Trans. Multim..