Business impact visualization for information security and compliance events

Business leaders face significant challenges from IT incidents that interfere with or pose imminent risk to more than one workgroup. Communication, coordination and monitoring are hindered by factors such as the IT incidents’ technical complexity and unfamiliarity, distributed ad-hoc response teams, competing demands for their time, nuanced business dependencies, the lack of reliable IT incident measures and a piecemeal toolset to overcome these challenges. This research proposes a dynamic visual system as a solution to overcome many of these challenges. Starting with a broad outline of improving the awareness and comprehension of security and compliance events for business leaders, this effort enlisted the assistance of seven experienced IT professionals in the Des Moines metropolitan area. A user-centered design methodology was developed that enabled these individuals to influence the selection of a problem space, explore related challenges, contribute to requirements definition and prioritization, review designs and, finally, test a prototype. The group consisted of leaders and senior technical staff working in various industries. At the end of the methodology, a group of unrelated IT professionals, with no prior knowledge, of the research was asked to perform an objective evaluation of the prototype. That evaluation is reported in this document and forms the basis of conclusions regarding the research hypothesis.

[1]  Joachim Karlsson,et al.  Software requirements prioritizing , 1996, Proceedings of the Second International Conference on Requirements Engineering.

[2]  Colin Ware,et al.  Visual Queries: The Foundation of Visual Thinking , 2005, Knowledge and Information Visualization.

[3]  Fred Volk,et al.  Proposing a multi-touch interface for intrusion detection environments , 2010, VizSec '10.

[4]  Alan Dix Chapter 3: Human-Computer Interaction and Web Design , 2004 .

[5]  Dan Diaper,et al.  Scenarios and task analysis , 2002, Interact. Comput..

[6]  Anita D'Amico,et al.  Information assurance visualizations for specific stages of situational awareness and intended uses: lessons learned , 2005, IEEE Workshop on Visualization for Computer Security, 2005. (VizSEC 05)..

[7]  G. Willis,et al.  Does Pretesting Make a Difference? An Experimental Test , 2004 .

[8]  William A. Pike,et al.  Putting Security in Context: Visual Correlation of Network Activity with Real-World Information , 2007, VizSEC.

[9]  Barry W. Boehm,et al.  Using Risk to Balance Agile and Plan-Driven Methods , 2003, Computer.

[10]  Thomas L. Saaty,et al.  Multicriteria Decision Making: The Analytic Hierarchy Process: Planning, Priority Setting, Resource Allocation , 1990 .

[11]  Gregory Conti,et al.  Countering network-level denial of information attacks using information visualization , 2006 .

[12]  Alberto J. Cañas,et al.  A TEORIA SUBJACENTE AOS MAPAS CONCEITUAIS E COMO ELABORÁ-LOS E USÁ-LOS * THE THEORY UNDERLYING CONCEPT MAPS AND HOW TO CONSTRUCT AND USE THEM , 2010 .

[13]  Tohru Ifukube,et al.  Maximum listening speeds for the blind , 2003 .

[14]  S. Donaldson,et al.  Understanding Self-Report Bias in Organizational Behavior Research , 2002 .

[15]  Wayne G. Lutters,et al.  The Work of Intrusion Detection: Rethinking the Role of Security Analysts , 2004, AMCIS.

[16]  Ben Shneiderman,et al.  The eyes have it: a task by data type taxonomy for information visualizations , 1996, Proceedings 1996 IEEE Symposium on Visual Languages.

[17]  Dan Diaper,et al.  Tasks for and tasks in human-computer interaction , 2006, Interact. Comput..

[18]  John R. Goodall,et al.  A user-centered look at glyph-based security visualization , 2005, IEEE Workshop on Visualization for Computer Security, 2005. (VizSEC 05)..

[19]  Paul E. Green,et al.  Conjoint Analysis in Marketing: New Developments with Implications for Research and Practice , 1990 .

[20]  John R. Goodall,et al.  Visualization is better! A comparative evaluation , 2009, 2009 6th International Workshop on Visualization for Cyber Security.

[21]  Michael Wurst,et al.  Collaborative Knowledge Visualization for Cross-Community Learning , 2005, Knowledge and Information Visualization.

[22]  Joachim Karlsson,et al.  Supporting the selection of software requirements , 1996, Proceedings of the 8th International Workshop on Software Specification and Design.

[23]  Ben Shneiderman,et al.  Readings in information visualization - using vision to think , 1999 .

[24]  Gregory T Sica,et al.  Bias in research studies. , 2006, Radiology.

[25]  Raanan Lipshitz,et al.  Converging themes in the study of decision making in realistic settings. , 1993 .

[26]  Joey F. George,et al.  Modern Systems Analysis and Design , 1996 .

[27]  M. Couper,et al.  Methods for Testing and Evaluating Survey Questionnaires , 2004 .

[28]  Sigmar-Olaf Tergan,et al.  Visualizing Knowledge and Information: An Introduction , 2005, Knowledge and Information Visualization.

[29]  Karl T. Ulrich,et al.  Product Design and Development , 1995 .

[30]  H. Margolis Visual explanations: Images and quantities, evidence and narrative , 1998 .

[31]  Tore Dybå,et al.  What Do We Know about Agile Software Development? , 2009, IEEE Software.

[32]  D. Norman Emotional design : why we love (or hate) everyday things , 2004 .

[33]  Jeffrey C. Carver,et al.  A visual analytic framework for exploring relationships in textual contents of digital forensics evidence , 2009, 2009 6th International Workshop on Visualization for Cyber Security.

[34]  Anita D. D'Amico,et al.  The Real Work of Computer Network Defense Analysts , 2007, VizSEC.

[35]  G. Klein,et al.  A recognition-primed decision (RPD) model of rapid decision making. , 1993 .

[36]  R. Hutton,et al.  Applied cognitive task analysis (ACTA): a practitioner's toolkit for understanding cognitive task demands. , 1998, Ergonomics.

[37]  Pin Ren Ensuring the continuing success of vizsec , 2006, VizSEC '06.

[38]  Yvonne Rogers,et al.  Interaction Design: Beyond Human-Computer Interaction , 2002 .

[39]  Daniel A. Keim,et al.  Large-Scale Network Monitoring for Visual Analysis of Attacks , 2008, VizSEC.

[40]  Deborah A. Frincke,et al.  A Multi-Phase Network Situational Awareness Cognitive Task Analysis , 2010, Inf. Vis..

[41]  Chris Horn,et al.  Visual analysis of goal-directed network defense decisions , 2011, VizSec '11.

[42]  M. Molhanec The Agile Methods - an Innovative Approach in the Project Management , 2007, 2007 30th International Spring Seminar on Electronics Technology (ISSE).

[43]  Cleo D. Redline,et al.  Testing Paper Self‐Administered Questionnaires: Cognitive Interview and Field Test Comparisons , 2004 .

[44]  T. DeMaio,et al.  Do Different Cognitive Interview Techniques Produce Different Results , 2004 .

[45]  Christina Wasylyshyn,et al.  Comprehension of Speech Presented at Synthetically Accelerated Rates: Evaluating Training and Practice Effects , 2010 .

[46]  Theresa J. DeMaio,et al.  Cognitive interviewing techniques: In the lab and in the field. , 1996 .

[47]  Siegfried Treu User Interface Evaluation: A Structured Approach , 1994 .

[48]  Shaun Moon,et al.  Visual correlation for situational awareness , 2005, IEEE Symposium on Information Visualization, 2005. INFOVIS 2005..

[49]  Iris Vessey,et al.  Cognitive Fit: A Theory‐Based Analysis of the Graphs Versus Tables Literature* , 1991 .

[50]  Dennis F. Galletta,et al.  Cognitive Fit: An Empirical Study of Information Acquisition , 1991, Inf. Syst. Res..

[51]  Roberta Calderwood,et al.  Critical decision method for eliciting knowledge , 1989, IEEE Trans. Syst. Man Cybern..

[52]  Ben Shneiderman,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction , 1998 .

[53]  Gary Klein,et al.  Making Sense of Sensemaking 1: Alternative Perspectives , 2006, IEEE Intelligent Systems.

[54]  W. Keith Edwards,et al.  Adapting Personas for Use in Security Visualization Design , 2007, VizSEC.

[55]  Karen A. Scarfone,et al.  Computer Security Incident Handling Guide , 2004 .

[56]  Chris North,et al.  Visualizing cyber security: Usable workspaces , 2009, 2009 6th International Workshop on Visualization for Cyber Security.

[57]  Lisa Young,et al.  A Taxonomy of Operational Cyber Security Risks , 2010 .

[58]  Polona Vilar,et al.  Designing the User Interface: Strategies for Effective Human-Computer Interaction (5th edition) , 2010, J. Assoc. Inf. Sci. Technol..

[59]  Robert F. Erbacher Visualization design for immediate high-level situational assessment , 2012, VizSec '12.

[60]  Patrick Hertzog Visualizations to improve reactivity towards security incidents inside corporate networks , 2006, VizSEC '06.

[61]  Stefano Foresti,et al.  Visual correlation of network alerts , 2006, IEEE Computer Graphics and Applications.

[62]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[63]  D. Hantula Sources of Power: How People Make Decisions , 2001 .

[64]  Lisa McGarthwaite,et al.  BioN: a novel interface for biological network visualization , 2008 .

[65]  Gary Klein,et al.  Working Minds: A Practitioner's Guide to Cognitive Task Analysis , 2006 .

[66]  Wayne G. Lutters,et al.  An Information Visualization Framework for Intrusion Detection , 2004, CHI EA '04.

[67]  E. Salas,et al.  Team decision making in complex environments. , 1993 .

[68]  Christopher J. Novak,et al.  2009 Data Breach Investigations Report , 2009 .

[69]  Jonathan D. Pfautz,et al.  Using cognitive engineering for system design and evaluation: A visualization aid for stability and support operations , 2006 .

[70]  John Terninko,et al.  Step by Step Qfd: Customer Driven Product Design , 1997 .

[71]  Angela P. Wetzel Internet, mail, and mixed‐mode surveys: The tailored design method , 2010 .

[72]  M. Polanyi Chapter 7 – The Tacit Dimension , 1997 .

[73]  Frederick G. Conrad,et al.  Data Quality in Cognitive Interviews: The Case of Verbal Reports , 2004 .

[74]  Stefano Foresti,et al.  VisAlert: From Idea to Product , 2007, VizSEC.

[75]  Austin Henderson,et al.  Interaction design: beyond human-computer interaction , 2002, UBIQ.

[76]  P. Beatty The Dynamics of Cognitive Interviewing , 2004 .

[77]  Kate Ehrlich,et al.  Nimble cybersecurity incident management through visualization and defensible recommendations , 2010, VizSec '10.

[78]  Tobias Ley,et al.  Emerging Perspectives on Judgment and Decision Research: Command Style and Team Performance in Dynamic Decision-Making Tasks , 2003 .