Results from levels 2/3 fusion implementations: issues, challenges, retrospectives, and perspectives for the future an annotated perspective

Even though the definition of the Joint Director of Laboratories (JDL) "fusion levels" were established in 1987, published 1991, revised in 1999 and 2004, the meaning, effects, control and optimization of interactions among the fusion levels have not as yet been fully explored and understood. Specifically, this is apparent from the abstract JDL definitions of "Levels 2/3 Fusion" - situation and threat assessment (SA/TA), which involve deriving relations among entities, e.g., the aggregation of object states (i.e., classification and location) in SA, while TA uses SA products to estimate/predict the impact of actions/interactions effects on situations taken by the participant entities involved. Given all the existing knowledge in the information fusion and human factors literature, (both prior to and after the introduction of "fusion levels" in 1987) there are still open questions remaining in regard to implementation of knowledge representation and reasoning methods under uncertainty to afford SA/TA. Therefore, to promote exchange of ideas and to illuminate the historical, current and future issues associated with Levels 2/3 implementations, leading experts were invited to present their respective views on various facets of this complex problem. This paper is a retrospective annotated view of the invited panel discussion organized by Ivan Kadar (first author), supported by John Salerno, in order to provide both a historical perspective of the evolution of the state-of-the-art (SOA) in higher-level "Levels 2/3" information fusion implementations by looking back over the past ten or more years (before JDL), and based upon the lessons learned to forecast where focus should be placed to further enhance and advance the SOA by addressing key issues and challenges. In order to convey the panel discussion to audiences not present at the panel, annotated position papers summarizing the panel presentation are included.

[1]  Jürg Kohlas,et al.  A Mathematical Theory of Hints , 1995 .

[2]  Dale A. Lambert.  A Unification of Sensor and Higher-Level Fusion , 2006, 2006 9th International Conference on Information Fusion.

[3]  Lotfi A. Zadeh,et al.  On the Validity of Dempster''s Rule of Combination of Evidence , 1979 .

[4]  Dale Lambert.  Automating cognitive routines , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[5]  Joachim Biermann.  Understanding Military Information Processing — An Approach to Support Intelligence in Defence and Security , 2009 .

[6]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[7]  Erik Blasch,et al.  Issues and challenges in resource management and its interaction with levels 2/3 fusion with applications to real-world problems: an annotated perspective , 2007, SPIE Defense + Commercial Sensing.

[8]  Anne-Laure Jousselme,et al.  Situation awareness and ability in coalitions , 2007, 2007 10th International Conference on Information Fusion.

[9]  James Llinas,et al.  Revisiting the JDL Data Fusion Model II , 2004 .

[10]  D. J. Barrett,et al.  Model-Data Fusion , 2003 .

[11]  Belur V. Dasarathy,et al.  Sensor fusion potential exploitation-innovative architectures and illustrative applications , 1997, Proc. IEEE.

[12]  Subrata Das.  Situation assessment in urban combat environments , 2005, 2005 7th International Conference on Information Fusion.

[13]  Allen M. Waxman,et al.  SeeCoast: persistent surveillance and automated scene understanding for ports and coastal areas , 2007, SPIE Defense + Commercial Sensing.

[14]  Dale Lambert.  Grand challenges of information fusion , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[15]  Joseph Y. Halpern,et al.  Characterizing Solution Concepts in Games Using Knowledge-Based Programs , 2006, IJCAI.

[16]  Joachim Biermann,et al.  From Unstructured to Structured Information in Military Intelligence - Some Steps to Improve Information Fusion , 2004 .

[17]  Carol McCann,et al.  The human in command : exploring the modern military experience , 2000 .

[18]  Joachim Biermann.  A Knowledge-Based Approach to Information Fusion for the Support of Military Intelligence , 2004 .

[19]  E. Ruspini.  The Logical Foundations of Evidential Reasoning (revised) , 1987 .

[20]  John D. Lowrance,et al.  Understanding evidential reasoning , 1992, Int. J. Approx. Reason..

[21]  G.A. Carpenter,et al.  Information fusion for image analysis: geospatial foundations for higher-level fusion , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[22]  Anne-Laure Jousselme,et al.  Interpreted systems for situation analysis , 2007, 2007 10th International Conference on Information Fusion.

[23]  Jr. Hartley Rogers.  Theory of Recursive Functions and Effective Computability , 1969 .

[24]  D. A. Lambert,et al.  Ubiquitous command and control , 2007, 1999 Information, Decision and Control. Data and Information Fusion Symposium, Signal Processing and Communications Symposium and Decision and Control Symposium. Proceedings (Cat. No.99EX251).

[25]  James Llinas,et al.  An introduction to multisensor data fusion , 1997, Proc. IEEE.

[26]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[27]  J. Scholz,et al.  A Dialectic for Network Centric Warfare , 2005 .

[28]  Don Perugini,et al.  Agents for logistics: a provisional agreement approach , 2006 .

[29]  Ivan Kadar.  Results from levels 2/3 fusion implementations: Issues, challenges, retrospectives and perspectives for the future - An annotated view , 2007, 2007 10th International Conference on Information Fusion.

[30]  I. Kadar.  Knowledge representation issues in perceptual reasoning managed situation assessment , 2005, 2005 7th International Conference on Information Fusion.

[31]  Alex M. Andrew,et al.  Uncertainty and Information: Foundations of Generalized Information Theory , 2006 .

[32]  Ulrich Schade,et al.  Towards Automatic Threat Recognition , 2006 .

[33]  Joseph Y. Halpern,et al.  Plausibility Measures: A User's Guide , 1995, UAI.

[34]  Bradley J. Rhodes,et al.  Probabilistic associative learning of vessel motion patterns at multiple spatial scales for maritime situation awareness , 2007, 2007 10th International Conference on Information Fusion.

[35]  Leon Sterling,et al.  Distributed information fusion agents , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[36]  Daphne Koller,et al.  Multi-Agent Influence Diagrams for Representing and Solving Games , 2001, IJCAI.

[37]  Alessandro Saffiotti,et al.  The Transferable Belief Model , 1991, ECSQARU.

[38]  James Reason,et al.  Human Error , 1990 .

[39]  Ivan Kadar,et al.  Perceptual reasoning in adaptive fusion processing , 2002, SPIE Defense + Commercial Sensing.

[40]  K. D. Lee,et al.  Hybrid model for intent estimation , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[41]  Michael Hinman,et al.  Building a Framework for Situation Awareness , 2004 .

[42]  David A. Fay,et al.  Learn-while-tracking, feature discovery and fusion of high-resolution radar range profiles , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[43]  C. Nowak,et al.  The semantic challenge for situation assessments , 2005, 2005 7th International Conference on Information Fusion.

[44]  Donald A. Norman,et al.  Some observations on mental models , 1987 .

[45]  Erik Blasch,et al.  Level 5: user refinement to aid the fusion process , 2003, SPIE Defense + Commercial Sensing.

[46]  M R Endsley,et al.  Level of automation effects on performance, situation awareness and workload in a dynamic control task. , 1999, Ergonomics.

[47]  Joachim Biermann.  HADES - A Knowledge-Based System for Message Interpretation and Situation Determination , 1998, IEA/AIE.

[48]  Mica R. Endsley,et al.  Design and Evaluation for Situation Awareness Enhancement , 1988 .

[49]  Ivan Kadar.  Perceptual reasoning managed situation assessment and adaptive fusion processing , 2001, SPIE Defense + Commercial Sensing.

[50]  Arthur P. Dempster,et al.  Upper and Lower Probabilities Induced by a Multivalued Mapping , 1967, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[51]  D. A. Lambert.  Situations for situation awareness , 2001 .

[52]  John J. Salerno.  Where's level 2/3 fusion - a look back over the past 10 years , 2007, 2007 10th International Conference on Information Fusion.

[53]  E. Blasch,et al.  Assembling a distributed fused information-based human-computer cognitive decision making tool , 2000, IEEE Aerospace and Electronic Systems Magazine.

[54]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[55]  Alan N. Steinberg,et al.  Revisions to the JDL data fusion model , 1999, Defense, Security, and Sensing.

[56]  Johan Schubert,et al.  Evidential force aggregation , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[57]  Judea Pearl,et al.  Reasoning with belief functions: An analysis of compatibility , 1990, Int. J. Approx. Reason..

[58]  Erik Blasch,et al.  Issues and Challenges in Situation Assessment (Level 2 Fusion) , 2006, J. Adv. Inf. Fusion.

[59]  Driss Kettani,et al.  A qualitative spatial model for information fusion and situation analysis , 2000, Proceedings of the Third International Conference on Information Fusion.

[60]  M. Bedworth,et al.  The Omnibus model: a new model of data fusion? , 2000 .

[61]  Patrick Maupin,et al.  Clustering-based control of discrimination power in target tracking applications , 2006 .

[62]  Mieczyslaw M. Kokar,et al.  SAWA: an assistant for higher-level fusion and situation awareness , 2005, SPIE Defense + Commercial Sensing.

[63]  Fred G. Wetzel.  Kognitive Psychologie : eine Einführung in die Psychologie der kognitiven Strukturen von Jean Piaget , 1980 .

[64]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[65]  Ronald R. Yager,et al.  Entropy and Specificity in a Mathematical Theory of Evidence , 2008, Classic Works of the Dempster-Shafer Theory of Belief Functions.

[66]  Lynn lfenster.  Data Fusion in Offensive and Defensive Information Operations , 2000 .

[67]  Ivan Kadar.  Issues in Adaptive and Automatic Information Fusion Resource Management , 2006, 2006 9th International Conference on Information Fusion.

[68]  Subrata Das,et al.  Spatiotemporal Clustering for Aggregating Hostile Units in Cluttered Environments , 2006, 2006 9th International Conference on Information Fusion.

[69]  Gerald M. Powell,et al.  Issues and challenges of knowledge representation and reasoning methods in situation assessment (Level 2 Fusion) , 2006, SPIE Defense + Commercial Sensing.