A visual analytic framework for data fusion in investigative intelligence

Intelligence analysis depends on data fusion systems to provide capabilities of detecting and tracking important objects, events, and their relationships in connection to an analytical situation. However, automated data fusion technologies are not mature enough to offer reliable and trustworthy information for situation awareness. Given the trend of increasing sophistication of data fusion algorithms and loss of transparency in data fusion process, analysts are left out of the data fusion process cycle with little to no control and confidence on the data fusion outcome. Following the recent rethinking of data fusion as human-centered process, this paper proposes a conceptual framework towards developing alternative data fusion architecture. This idea is inspired by the recent advances in our understanding of human cognitive systems, the science of visual analytics, and the latest thinking about human-centered data fusion. Our conceptual framework is supported by an analysis of the limitation of existing fully automated data fusion systems where the effectiveness of important algorithmic decisions depend on the availability of expert knowledge or the knowledge of the analyst’s mental state in an investigation. The success of this effort will result in next generation data fusion systems that can be better trusted while maintaining high throughput.

[1]  Mieczyslaw M. Kokar,et al.  High-level information fusion and situation awareness , 2009, Inf. Fusion.

[2]  John J. Salerno.  Information fusion: a high-level architecture overview , 2002, Proceedings of the Fifth International Conference on Information Fusion. FUSION 2002. (IEEE Cat.No.02EX5997).

[3]  Sushant S. Khopkar,et al.  Data association and graph analytical processing of hard and soft intelligence data , 2013, Proceedings of the 16th International Conference on Information Fusion.

[4]  Belur V. Dasarathy,et al.  Sensor fusion potential exploitation-innovative architectures and illustrative applications , 1997, Proc. IEEE.

[5]  Dale Lambert.  Grand challenges of information fusion , 2003, Sixth International Conference of Information Fusion, 2003. Proceedings of the.

[6]  Stuart C. Shapiro,et al.  Towards hard+soft data fusion: Processing architecture and implementation for the joint fusion and analysis of hard and soft intelligence data , 2012, 2012 15th International Conference on Information Fusion.

[7]  Lawrence W Barsalou,et al.  Simulation, situated conceptualization, and prediction , 2009, Philosophical Transactions of the Royal Society B: Biological Sciences.

[8]  Pascal Vasseur,et al.  Introduction to Multisensor Data Fusion , 2005, The Industrial Information Technology Handbook.

[9]  Andrew Howes,et al.  Too Much, Too Little or Just Right: Designing Data Fusion for Situation Awareness , 2004 .

[10]  James Llinas,et al.  High Level Information Fusion (HLIF): Survey of models, issues, and grand challenges , 2012, IEEE Aerospace and Electronic Systems Magazine.

[11]  David L. Hall,et al.  Dirty Secrets in Multisensor Data Fusion , 2001 .

[12]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[13]  John T. Stasko,et al.  Distributed Cognition as a Theoretical Framework for Information Visualization , 2008, IEEE Transactions on Visualization and Computer Graphics.

[14]  Ilknur Icke,et al.  Visual Analytics: A Multifaceted Overview , 2009 .

[15]  Ann M. Bisantz,et al.  Identification of human-interaction touch points for intelligence analysis information fusion systems , 2011, 14th International Conference on Information Fusion.

[16]  Rakesh Nagi,et al.  A fuzzy graph matching approach in intelligence analysis and maintenance of continuous situational awareness , 2014, Inf. Fusion.

[17]  Stuart C. Shapiro,et al.  Tractor: A framework for soft information fusion , 2010, 2010 13th International Conference on Information Fusion.

[18]  James Llinas,et al.  Handbook of Multisensor Data Fusion , 2001 .

[19]  David S. Ebert,et al.  Data Transformations and Representations for Computation and Visualization , 2009, Inf. Vis..

[20]  Ray Jackendo,et al.  Languages of the Mind , 1992 .

[21]  Christine D. Wilson,et al.  Grounding conceptual knowledge in modality-specific systems , 2003, Trends in Cognitive Sciences.

[22]  Richard L. Tutwiler,et al.  Hard sensor fusion for COIN inspired situation awareness , 2011, 14th International Conference on Information Fusion.

[23]  J. Nacht.  Cognition Exploring The Science Of The Mind , 2016 .

[24]  Hamish Cunningham,et al.  GATE-a General Architecture for Text Engineering , 1996, COLING.

[25]  Cynthia Breazeal,et al.  Cognition as coordinated non-cognition , 2007, Cognitive Processing.

[26]  James Llinas,et al.  Handbook of Multisensor Data Fusion : Theory and Practice, Second Edition , 2008 .

[27]  James Llinas,et al.  Multisensor Data Fusion , 1990 .

[28]  Stuart C. Shapiro,et al.  Natural language understanding for soft information fusion , 2013, Proceedings of the 16th International Conference on Information Fusion.

[29]  L. Barsalou,et al.  The situated nature of concepts. , 2006, The American journal of psychology.

[30]  J. Carlos.  Languages of the Mind , 1995 .

[31]  Mica R. Endsley,et al.  Design and Evaluation for Situation Awareness Enhancement , 1988 .

[32]  Dale A. Lambert,et al.  A blueprint for higher-level fusion systems , 2009, Inf. Fusion.

[33]  David L. Hall,et al.  Human-Centered Information Fusion: Artech House Electronic Warfare Library , 2010 .

[34]  Leonid I. Perlovsky,et al.  Language and cognition , 2009, Neural Networks.

[35]  Matthias Scheutz,et al.  Symbol Grounding and the Origin of Language , 2003 .

[36]  Leonid I. Perlovsky,et al.  Conundrum of Combinatorial Complexity , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[37]  Robert L. Goldstone,et al.  Reuniting perception and conception , 1998, Cognition.

[38]  Lawrence W. Barsalou,et al.  The Cambridge Handbook of Psycholinguistics: The Human Conceptual System , 2012 .

[39]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[40]  Lawrence W. Barsalou,et al.  Concepts and meaning , 1993 .

[41]  Pramod K. Varshney,et al.  Multisensor Data Fusion , 1997, IEA/AIE.

[42]  Leonid I. Perlovsky,et al.  Cognitive high level information fusion , 2007, Inf. Sci..

[43]  L. Barsalou,et al.  Perceptual simulation in conceptual combination: evidence from property generation. , 2009, Acta psychologica.

[44]  Mica R. Endsley,et al.  Toward a Theory of Situation Awareness in Dynamic Systems , 1995, Hum. Factors.

[45]  J. Mehler,et al.  LANGUAGE AND COGNITION , 1998 .

[46]  Rakesh Nagi,et al.  The graph association problem: Mathematical models and a lagrangian heuristic , 2013 .