Measuring and visualizing cyber threat intelligence quality

The very raison d’être of cyber threat intelligence (CTI) is to provide meaningful knowledge about cyber security threats. The exchange and collaborative generation of CTI by the means of sharing platforms has proven to be an important aspect of practical application. It is evident to infer that inaccurate, incomplete, or outdated threat intelligence is a major problem as only high-quality CTI can be helpful to detect and defend against cyber attacks. Additionally, while the amount of available CTI is increasing it is not warranted that quality remains unaffected. In conjunction with the increasing number of available CTI, it is thus in the best interest of every stakeholder to be aware of the quality of a CTI artifact. This allows for informed decisions and permits detailed analyses. Our work makes a twofold contribution to the challenge of assessing threat intelligence quality. We first propose a series of relevant quality dimensions and configure metrics to assess the respective dimensions in the context of CTI. In a second step, we showcase the extension of an existing CTI analysis tool to make the quality assessment transparent to security analysts. Furthermore, analysts’ subjective perceptions are, where necessary, included in the quality assessment concept.

[1]  Diane M. Strong,et al.  Beyond Accuracy: What Data Quality Means to Data Consumers , 1996, J. Manag. Inf. Syst..

[2]  Francisco Herrera,et al.  Distinguishing between facts and opinions for sentiment analysis: Survey and challenges , 2018, Inf. Fusion.

[3]  V. Villagrá,et al.  Leveraging cyber threat intelligence for a dynamic risk framework , 2019, International Journal of Information Security.

[4]  Tim Ring Threat intelligence: why people don't share , 2014 .

[5]  Carlo Batini,et al.  Data and Information Quality , 2016, Data-Centric Systems and Applications.

[6]  Veda C. Storey,et al.  A Framework for Analysis of Data Quality Research , 1995, IEEE Trans. Knowl. Data Eng..

[7]  Florian Skopik,et al.  A problem shared is a problem halved: A survey on the dimensions of collective cyber defense through security information sharing , 2016, Comput. Secur..

[8]  Bernd Grobauer,et al.  Mining Attributed Graphs for Threat Intelligence , 2017, CODASPY.

[9]  Christophe Rosenberger,et al.  Local user-centric identity management , 2014, Journal of Trust Management.

[10]  Ruth Breu,et al.  Data Quality Challenges and Future Research Directions in Threat Intelligence Sharing Practice , 2016, WISCS@CCS.

[11]  Harry Hochheiser,et al.  Research Methods for Human-Computer Interaction , 2008 .

[12]  Günther Pernul,et al.  Graph-based visual analytics for cyber threat intelligence , 2018, Cybersecurity.

[13]  Jürgen Umbrich,et al.  Quality Assessment and Evolution of Open Data Portals , 2015, 2015 3rd International Conference on Future Internet of Things and Cloud.

[14]  Wiem Tounsi,et al.  A survey on technical threat intelligence in the age of sophisticated cyber attacks , 2018, Comput. Secur..

[15]  E. Ziegel Juran's Quality Control Handbook , 1988 .

[16]  Thomas Redman,et al.  Data quality for the information age , 1996 .

[17]  Adam Doupé,et al.  Matched and Mismatched SOCs: A Qualitative Study on Security Operations Center Issues , 2019, CCS.

[18]  Carlo Batini,et al.  Methodologies for data quality assessment and improvement , 2009, CSUR.

[19]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[20]  Richard Y. Wang,et al.  Anchoring data quality dimensions in ontological foundations , 1996, CACM.

[21]  Ruth Breu,et al.  Towards a Maturity Model for Inter-Organizational Cyber Threat Intelligence Sharing: A Case Study of Stakeholders' Expectations and Willingness to Share , 2018 .

[22]  Audun Jøsang,et al.  A survey of trust and reputation systems for online service provision , 2007, Decis. Support Syst..

[23]  Carlo Batini,et al.  The Many Faces of Information and their Impact on Information Quality , 2012, ICIQ.

[24]  L. Dandurand,et al.  Towards improved cyber security information sharing , 2013, 2013 5th International Conference on Cyber Conflict (CYCON 2013).

[25]  Richard Y. Wang,et al.  Data Quality Assessment , 2002 .

[26]  Günther Pernul,et al.  A comparative analysis of incident reporting formats , 2018, Comput. Secur..

[27]  Ruth Breu,et al.  Threat Intelligence Sharing Platforms: An Exploratory Study of Software Vendors and Research Perspectives , 2017, Wirtschaftsinformatik.

[28]  Marcus Kaiser,et al.  How to Measure Data Quality? - A Metric-Based Approach , 2007, ICIS.

[29]  Carlo Batini,et al.  Data and Information Quality , 2016, Data-Centric Systems and Applications.

[30]  Günther Pernul,et al.  Reusable components for online reputation systems , 2015, Journal of Trust Management.

[31]  Sarah Brown,et al.  On the Design of a Cyber Security Data Sharing System , 2014, WISCS '14.