Performance Evaluation for Automated Threat Detection

We have developed a performance evaluation laboratory (PE Lab) to assess automated technologies that fuse fragmentary, partial information about individuals’ activities to detect modeled terrorist threat individuals, groups, and events whose evidence traces are embedded in a background dominated by evidence from similarly modeled non-threat phenomena. We have developed the PE Lab’s main components–a test dataset generator and a hypothesis scorer–to address two key challenges of counter-terrorism threat detection performance evaluation: 2 Acquiring adequate test data to support systematic experimentation; and 2 Scoring structured hypotheses that reflect modeled threat objects’ attribute values and inter-relationships.