Evaluation of a workshop to capture knowledge from subject matter experts in maritime surveillance

This paper reports on a brainstorming workshop with subject matter experts that aimed at identifying potential anomalies for world wide maritime surveillance. An extensive description of the workshop process and output is presented. The evaluation consists of two parts. First, the output of the workshop is compared with the output of a similar workshop as presented in Roy [1]. Secondly, the methodological differences between capturing know-how of subject matter experts in a workshop versus observing daily work of operators, as applied in Nilsson et al [2], is discussed. The conclusions are that brainstorming workshops and field studies should be combined to compensate for their weaknesses. Also, the outcomes of both methods heavily depend on the selection of operators to be observed, respectively the selection of subject matter experts invited. A final recommendation is that multiple studies and workshops are needed until no ‘new’ anomalies are identified.

[1]  Fabio Roli,et al.  Intrusion detection in computer networks by a modular ensemble of one-class classifiers , 2008, Inf. Fusion.

[2]  Jung-Min Park,et al.  An overview of anomaly detection techniques: Existing solutions and latest technological trends , 2007, Comput. Networks.

[3]  Pontus Svenson,et al.  SMARTracIn: a concept for spoof resistant tracking of vessels and detection of adverse intentions , 2009, Defense + Commercial Sensing.

[4]  Robert M. Davison,et al.  How a silver bullet may lose its shine , 2003, CACM.

[5]  Peter Jackson,et al.  Introduction to expert systems , 1986 .

[6]  Tom Ziemke,et al.  Extracting rules from expert operators to support situation awareness in maritime surveillance , 2008, 2008 11th International Conference on Information Fusion.

[7]  C. Cassell,et al.  Essential guide to qualitative methods in organizational research , 2004 .

[8]  Jean Roy,et al.  Anomaly detection in the maritime domain , 2008, SPIE Defense + Commercial Sensing.

[9]  Michael Davenport,et al.  Information Visualization: The State of the Art for Maritime Domain Awareness , 2006 .

[10]  Allen M. Waxman,et al.  Associative Learning of Vessel Motion Patterns for Maritime Situation Awareness , 2006, 2006 9th International Conference on Information Fusion.

[11]  Starr Roxanne Hiltz,et al.  An Assessment of Group Support Systems Experimental Research: Methodology and Results , 1998, J. Manag. Inf. Syst..

[12]  Northrop Grumman,et al.  Recommended Requirements Gathering Practices , 2002 .

[13]  Metin Balci,et al.  Towards Global Maritime Domain Awareness - "Recent Developments and Challenges" , 2006, 2006 9th International Conference on Information Fusion.

[14]  Starr Roxanne Hiltz,et al.  An assessment of group support systems research: results , 1999, Proceedings of the 32nd Annual Hawaii International Conference on Systems Sciences. 1999. HICSS-32. Abstracts and CD-ROM of Full Papers.

[15]  K. DeWalt,et al.  Participant Observation: A Guide for Fieldworkers , 2001 .

[16]  S. Fagerhaugh,et al.  Participant Observation , 1979 .

[17]  A. Vidich,et al.  A Comparison of Participant Observation and Survey Data , 1955 .