Mission Specific Embedded Training Using Mixed Reality

Abstract : High risk special operations often require that the team is trained in advance in an environment similar to the target area. It would be ideal if a replica could be built and the team could be trained in the operational environment. However, it is usually difficult to do so in a timely manner, especially when intelligence constantly changes. Traditionally a scaled-down model of the target area is built instead. Mixed reality based embedded training with the ability to rapidly build and improve the model is a useful tool that can meet these challenges. Several research groups have considered augmented reality (AR) or mixed reality as a training tool for military operations in urban terrain. Our group has developed the Battlefield Augmented Reality System (BARS)(trademark), which can be used for a variety of applications, such as situation awareness as well as embedded training. We have since developed a new version of the system that makes use of the state-of-the-art techniques to build missionspecific training systems in a timely manner. While there is still debate on how effective a current AR situation awareness system is in an actual combat situation, we have seen increased interest in AR training among Marines, Air Force special forces and others. In this paper, a mission specific embedded military training system using mixed reality is presented. The methods for building environmental models and embedding synthetic characters are described. A set of tools has been assembled to build mission specific models. These models together with a modular physical setting can be integrated into the training system which provides a mixture of virtual objects and physical objects. This mixed environment provides a more realistic training than a pure virtual environment, and more flexible training than pure physical settings.