Automated Support for AARs: Exploiting Communication to Assess Team Performance

The After Action Review (AAR) process provides a powerful methodology that in the context of training maximizes the benefits of exercises by enabling a unit to learn from experience by systematically reflecting on their strengths and weaknesses. We have developed a tool that supports the AAR process, essentially extending an Observer Controller’s (O/C) reach automatically. This tool was developed with two training contexts in mind: live STX lane convoy training at the National Training Center (NTC) and simulated convoy training using DARWARS Ambush! at the Mission Support Training Facility at Fort Lewis. At NTC, live radio communication is captured during training, while with Ambush! communication using voice over IP (VOIP) is recorded. The tool automatically converts recorded speech to text and then analyzes the text, using advanced statistical machine learning technologies, to determine a unit’s performance and identify critical incidents, leading indicators, and other training events that could be included in an AAR. We worked closely with Subject Matter Experts (SMEs) to derive the important dimensions of performance allowing the tool to support a wide range of O/C and commander AARs. The tool rates a unit on several scales based on a mission essential task list (METL), including command and control, situation understanding, use of standard operating procedures (SOPs), and battle drills. For each rating scale, the tool selects appropriate training events that reflect the unit’s range of performance from untrained through practiced to trained. The tool’s interface makes it easy to spot performance weaknesses at a glance and then to drill down to understand these weaknesses by listening to the relevant radio communication. The tool also enables commanders to create a custom AAR by selecting events of interest and the associated radio communication and then adding their own comments.