Supporting the Analytical Reasoning Process in Maritime Anomaly Detection: Evaluation and Experimental Design

Despite the growing number of systems providing visual analytic support for investigative analysis, few empirical studies include investigations on the analytical reasoning process that needs to be supported. In this paper, we present an approach to evaluate the ability of certain visual representations from an integrated visual-computational environment to support the completion of representative tasks. The problem area studied is the detection and identification of anomalous vessels and situations while monitoring maritime traffic data. This paper presents: (1) a brief review of current evaluation methodologies within information visualization and visual analytics, (2) an analysis of operator's analytical reasoning process (derived from field work in maritime control centers and a literature review on analytical reasoning theories), (3) a list of representative tasks for usability evaluation and (4) an approach to evaluate the use of normal behavioral models representations during the detection process.

[1]  Steven K. Feiner,et al.  Visual task characterization for automated visual discourse synthesis , 1998, CHI.

[2]  Jean Roy,et al.  Anomaly detection in the maritime domain , 2008, SPIE Defense + Commercial Sensing.

[3]  John Stasko Evaluating Information Visualizations : Issues and Opportunities , 2006 .

[4]  M. Sheelagh T. Carpendale,et al.  Evaluating Information Visualizations , 2008, Information Visualization.

[5]  Caroline Tobon Usability testing for improving interactive geovisualization techniques , 2002 .

[6]  Mary Czerwinski,et al.  Empirical evaluation of information visualizations: an introduction , 2000, Int. J. Hum. Comput. Stud..

[7]  Kristin A. Cook,et al.  Illuminating the Path: The Research and Development Agenda for Visual Analytics , 2005 .

[8]  P. Pirolli,et al.  The Sensemaking Process and Leverage Points for Analyst Technology as Identified Through Cognitive Task Analysis , 2007 .

[9]  Menno-Jan Kraak,et al.  Defining visualization operations for temporal cartographic animation design , 2002 .

[10]  Mark R. Morelande,et al.  Statistical analysis of motion patterns in AIS Data: Anomaly detection and motion prediction , 2008, 2008 11th International Conference on Information Fusion.

[11]  Jean Scholtz,et al.  Metrics for evaluating human information interaction systems , 2006, Interact. Comput..

[12]  Alan M. MacEachren,et al.  Evaluating the usability of visualization methods in an exploratory geovisualization environment , 2006, Int. J. Geogr. Inf. Sci..

[13]  Clayton Lewis,et al.  A problem-oriented classification of visualization techniques , 1990, Proceedings of the First IEEE Conference on Visualization: Visualization `90.

[14]  Jean Scholtz,et al.  Advancing User-Centered Evaluation of Visual Analytic Environments through Contests , 2009, Inf. Vis..

[15]  Michela Bertolotto,et al.  Exploratory spatio-temporal data mining and visualization , 2007, J. Vis. Lang. Comput..

[16]  Urška Demšar Combining Formal and Exploratory Methods for Evaluation of an Exploratory Geovisualization Application in a Low-Cost Usability Experiment , 2007 .

[17]  Bottoni Paolo,et al.  Working Conference on Advanced Visual Interfaces , 2008, AVI 2008.

[18]  Tom Ziemke,et al.  VISAD: an interactive and visual analytical tool for the detection of behavioral anomalies in maritime traffic data , 2009, Defense + Commercial Sensing.

[19]  Gary Klein,et al.  Making Sense of Sensemaking 1: Alternative Perspectives , 2006, IEEE Intelligent Systems.

[20]  Emilie M. Roth,et al.  Predicting Vulnerabilities in Computer-Supported Inferential Analysis under Data Overload , 2001, Cognition, Technology & Work.

[21]  P. Pirolli Information Foraging Theory: Adaptive Interaction with Information , 2007 .

[22]  Tom Ziemke,et al.  Reasoning about anomalies: a study of the analytical process of detecting and identifying anomalous behavior in maritime traffic data , 2009, Defense + Commercial Sensing.

[23]  Göran Falkman,et al.  Evaluating the Usability of Visualizations of Normal Behavioral Models for Analytical Reasoning , 2010, 2010 Seventh International Conference on Computer Graphics, Imaging and Visualization.

[24]  Catherine Plaisant,et al.  The challenge of information visualization evaluation , 2004, AVI.

[25]  Alan J. Dix,et al.  Human Aspects of Visualization , 2009, HCIV.

[26]  D T Boumpas Editor's Foreword , 1998 .

[27]  Jean Scholtz,et al.  Beyond Usability: Evaluation Aspects of Visual Analytic Environments , 2006, 2006 IEEE Symposium On Visual Analytics Science And Technology.

[28]  James B. Kraiman,et al.  Automated anomaly detection processor , 2002, SPIE Defense + Commercial Sensing.