Measuring the worthiness of situation assessment

Situation Assessment (SA) modeling has many instances of development, but there has yet to be a comprehensive set of metrics used for performance evaluation. The amount of data being presented and displayed to the analyst is overwhelming — to a point that in many cases they are missing the salient or key activities of interest (AOI). Analysts are spending the majority of their time filtering through the data rather than performing analysis. To aid the user (high-level information fusion), we explore the nature of how we can rank various activities based on their impact and threat. We develop a information fusion SA reference model over SA metrics of confidence (precision, recall), accuracy, activities of interest, timeliness and throughput to determine the intent and assessment of situation activities.

[1]  Erik Blasch,et al.  Level 5 (User Refinement) issues supporting Information Fusion Management , 2006, 2006 9th International Conference on Information Fusion.

[2]  Alfred Morris,et al.  Information for decision-making , 1984 .

[3]  Jean Roy,et al.  Concepts, Models, and Tools for Information Fusion , 2007 .

[4]  Erik P. Blasch,et al.  Fusion metrics for dynamic situation analysis , 2004, SPIE Defense + Commercial Sensing.

[5]  D. A. Lambert.  Situations for situation awareness , 2001 .

[6]  George P. Tadda.  Measuring performance of Cyber situation awareness systems , 2008, 2008 11th International Conference on Information Fusion.

[7]  Gerald M. Powell,et al.  Issues and challenges of knowledge representation and reasoning methods in situation assessment (Level 2 Fusion) , 2006, SPIE Defense + Commercial Sensing.

[8]  Erik Blasch,et al.  Situation, impact, and user refinement , 2003, SPIE Defense + Commercial Sensing.

[9]  Michael Hinman,et al.  Building a Framework for Situation Awareness , 2004 .

[10]  Eloi Bosse,et al.  The Cognitive Costs and Benefits of Automation , 2003 .

[11]  Erik Blasch,et al.  Implication of culture: User roles in information Fusion for enhanced situational understanding , 2009, 2009 12th International Conference on Information Fusion.

[12]  Jose B. Cruz,et al.  Game Theoretic Approach to Threat Prediction and Situation Awareness , 2006, 2006 9th International Conference on Information Fusion.

[13]  Moises Sudit,et al.  Issues and challenges in higher level fusion: Threat/impact assessment and intent modeling (a panel summary) , 2010, 2010 13th International Conference on Information Fusion.

[14]  John J. Salerno,et al.  Realizing situation awareness within a cyber environment , 2006, SPIE Defense + Commercial Sensing.

[15]  Jean Roy,et al.  Threat evaluation for impact assessment in situation analysis systems , 2002, SPIE Defense + Commercial Sensing.

[16]  Erik Blasch,et al.  Physio-associative temporal sensor integration , 1998, Defense, Security, and Sensing.

[17]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[18]  John J. Salerno,et al.  Ranking Activities Based on Their Impact and Threat , 2009 .

[19]  E. Blasch,et al.  Sensor Management Fusion Using Operating Conditions , 2008, 2008 IEEE National Aerospace and Electronics Conference.

[20]  John J. Salerno,et al.  Evaluating algorithmic techniques in supporting situation awareness , 2005, SPIE Defense + Commercial Sensing.

[21]  Leonid I. Perlovsky,et al.  Cognitive high level information fusion , 2007, Inf. Sci..

[22]  Erik Blasch,et al.  High Level Information Fusion developments, issues, and grand challenges: Fusion 2010 panel discussion , 2010, 2010 13th International Conference on Information Fusion.

[23]  E. Blasch,et al.  Assembling a distributed fused information-based human-computer cognitive decision making tool , 2000, IEEE Aerospace and Electronic Systems Magazine.

[24]  John Salerno.  Measuring situation assessment performance through the activities of interest score , 2008, 2008 11th International Conference on Information Fusion.

[25]  James Llinas,et al.  Revisiting the JDL Data Fusion Model II , 2004 .

[26]  S. Plano,et al.  Proactive decision fusion for site security , 2005, 2005 7th International Conference on Information Fusion.

[27]  S. Plano,et al.  DFIG level 5 (user refinement) issues supporting situational assessment reasoning , 2005, 2005 7th International Conference on Information Fusion.

[28]  Richard Breton,et al.  FUSION IN SUPPORT OF DYNAMIC HUMAN DECISION MAKING , 1999 .

[29]  Erik Blasch,et al.  User information fusion decision making analysis with the C-OODA model , 2011, 14th International Conference on Information Fusion.

[30]  John J. Salerno.  Resource Management: A Necessary and Integral Component to Any Level 2/3 Fusion Capability , 2006, 2006 9th International Conference on Information Fusion.

[31]  Erik Blasch,et al.  Three-dimensional receiver operating characteristic (ROC) trajectory concepts for the evaluation of target recognition algorithms faced with the unknown target detection problem , 1999, Defense, Security, and Sensing.

[32]  David S. Fadok.  Air Power's Quest for Strategic Paralysis , 1995 .

[33]  Erik Blasch,et al.  Level 5: user refinement to aid the fusion process , 2003, SPIE Defense + Commercial Sensing.