TRACE: A Stigmergic Crowdsourcing Platform for Intelligence Analysis

Crowdsourcing has become a frequently adopted approach to solving various tasks from conducting surveys to designing products. In the field of reasoning-support, however, crowdsourcing-related research and application have not been extensively implemented. Reasoning-support is essential in intelligence analysis to help analysts mitigate various cognitive biases, enhance deliberation, and improve report writing. In this paper, we propose a novel approach to designing a crowdsourcing platform that facilitates stigmergic coordination, awareness, and communication for intelligence analysis. We have partly materialized our proposal in the form of a crowdsourcing system which supports intelligence analysis: TRACE (Trackable Reasoning and Analysis for Collaboration and Evaluation). We introduce several stigmergic approaches integrated into TRACE and discuss the potential experimentation of these approaches. We also explain the design implications for further development of TRACE and similar crowdsourcing systems to support reasoning.

[1]  Darrell M. Hull,et al.  Negotiation of meaning and co-construction of knowledge: An experimental analysis of asynchronous online instruction , 2009, Comput. Educ..

[2]  Christer Carlsson,et al.  Past, present, and future of decision support technology , 2002, Decis. Support Syst..

[3]  S. Frederick.  Journal of Economic Perspectives—Volume 19, Number 4—Fall 2005—Pages 25–42 Cognitive Reflection and Decision Making , 2022 .

[4]  Katrin Weller,et al.  Tag Gardening for Folksonomy Enrichment and Maintenance , 2008, Webology.

[5]  Martin Schader,et al.  Crowdsourcing Information Systems - Definition, Typology, and Design , 2012, ICIS.

[6]  Kwan Yi,et al.  Harnessing collective intelligence in social tagging using Delicious , 2012, J. Assoc. Inf. Sci. Technol..

[7]  Carsten Eickhoff,et al.  Cognitive Biases in Crowdsourcing , 2018, WSDM.

[8]  Daren C. Brabham.  MOVING THE CROWD AT THREADLESS , 2010 .

[9]  R. Nickerson.  Confirmation Bias: A Ubiquitous Phenomenon in Many Guises , 1998 .

[10]  Stefan Dietze,et al.  A taxonomy of microtasks on the web , 2014, HT.

[11]  Oded Nov,et al.  Volunteer computing: a model of the factors determining contribution to community-based scientific research , 2010, WWW '10.

[12]  Michael S. Bernstein,et al.  Mechanical Novel: Crowdsourcing Complex Work through Reflection and Revision , 2016, CSCW.

[13]  D. Kahneman,et al.  Heuristics and Biases: The Psychology of Intuitive Judgment , 2002 .

[14]  Ralph H. Sprague,et al.  A Framework for the Development of Decision Support Systems , 1993 .

[15]  Chei Sian Lee,et al.  Perceptions, quality and motivational needs in image tagging human computation games , 2011, J. Inf. Sci..

[16]  Stephen Coulthart,et al.  An Evidence-Based Evaluation of 12 Core Structured Analytic Techniques , 2017 .

[17]  András Kornai,et al.  Edit Wars in Wikipedia , 2011, 2011 IEEE Third Int'l Conference on Privacy, Security, Risk and Trust and 2011 IEEE Third Int'l Conference on Social Computing.

[18]  A. Furnham,et al.  A literature review of the anchoring effect , 2011 .

[19]  P. Resnick,et al.  Encouraging Commitment in Online Communities , 2011 .

[20]  Kerry Raymond,et al.  General theory of stigmergy: Modelling stigma semantics , 2014, Cognitive Systems Research.

[21]  Efraim Turban,et al.  What can crowdsourcing do for decision support? , 2014, Decis. Support Syst..

[22]  Lars Rune Christensen,et al.  The logic of practices of stigmergy: representational artifacts in architectural design , 2008, CSCW.

[23]  Aniket Kittur,et al.  CrowdForge: crowdsourcing complex work , 2011, UIST.

[24]  Artner Stephen,et al.  Assessing the Value of Structured Analytic Techniques in the U.S. Intelligence Community , 2016 .

[25]  J. Kruger,et al.  Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. , 1999, Journal of personality and social psychology.

[26]  Carsten S. Østerlund,et al.  Folksonomies to Support Coordination and Coordination of Folksonomies , 2018, Computer Supported Cooperative Work (CSCW).

[27]  A. Tversky,et al.  Subjective Probability: A Judgment of Representativeness , 1972 .

[28]  Michael S. Bernstein,et al.  The future of crowd work , 2013, CSCW.