Exploiting collective knowledge with three-way decision theory: Cases from the questionnaire-based research

Two methods are proposed for collective knowledge extraction from questionnaires with ordinal scales and dichotomous questions.Both methods are based on a three-way decision procedure and a statistical method aimed at attaining statistical significance of the above decision. One method is aimed at giving an (absolute) assessment of objects according to a given criterion and the other one at producing a relative ranking of the objects. A criterion can be related to one or more questionnaire items (usually questions or statements). In this latter case a method to compose ordinal items in aggregate scores is also given. The paper also presents two various case studies that illustrate the methods and give motivations for their application in different domains where the knowledge of a community or any distributed group of experts can be externalized (in terms of users' perceptions, attitudes, opinions, choices) with a structured closed-ended questionnaire. The contribution proposes statistical procedures to extract collective knowledge from surveys.An original method is proposed to compose ordinal variables together.One Three-Way method is proposed to assess objects according to ordinal variables.One Three-Way method is proposed to rank objects according to ordinal variables.The methods have been validated in two real case studies of complex decision making.

[1]  Aniket Kittur,et al.  CrowdForge: crowdsourcing complex work , 2011, UIST.

[2]  Jan Marco Leimeister,et al.  Rating Scales for Collective Intelligence in Innovation Communities: Why Quick and Easy Decision Making Does Not Get it Right , 2010, ICIS.

[3]  P. Herr.  Priming Price: Prior Knowledge and Context Effects , 1989 .

[4]  Yiyu Yao,et al.  Rough Sets and Three-Way Decisions , 2015, RSKT.

[5]  Yiyu Yao,et al.  An Outline of a Theory of Three-Way Decisions , 2012, RSCTC.

[6]  Eelko K R E Huizingh,et al.  Applied Statistics with SPSS , 2007 .

[7]  Judy Robertson,et al.  Likert-type scales, statistical methods, and effect sizes , 2012, Commun. ACM.

[8]  Yiyu Yao,et al.  Advances in three-way decisions and granular computing , 2016, Knowl. Based Syst..

[9]  S S Stevens,et al.  On the Theory of Scales of Measurement. , 1946, Science.

[10]  Tommi Kärkkäinen,et al.  Expert-based versus citation-based ranking of scholarly and scientific publication channels , 2016, J. Informetrics.

[11]  E. Bonabeau.  Decisions 2.0: the power of collective intelligence , 2009 .

[12]  Federico Cabitza,et al.  Exploiting the Collective Knowledge of Communities of Experts , 2015, KMIS.

[13]  Timothy J Buckley,et al.  Development of a questionnaire to assess worker knowledge, attitudes and perceptions underlying dermal exposure. , 2006, Scandinavian journal of work, environment & health.

[14]  H. Ruchlin,et al.  The Efficacy of Second-Opinion Consultation Programs: A Cost–Benefit Perspective , 1982, Medical care.

[15]  Carla Simone,et al.  Investigating the role of a Web-based tool to promote collective knowledge in medical communities , 2012 .

[16]  Yiyu Yao,et al.  Granular Computing and Sequential Three-Way Decisions , 2013, RSKT.

[17]  Aniket Kittur,et al.  CrowdForge: crowdsourcing complex work , 2011, CHI Extended Abstracts.

[18]  Federico Cabitza,et al.  Questionnaires in the design and evaluation of community-oriented technologies , 2017, Int. J. Web Based Communities.

[19]  Panos Markopoulos,et al.  Powerful and consistent analysis of likert-type rating scales , 2010, CHI.

[20]  Yiyu Yao,et al.  Three-Way Decision: An Interpretation of Rules in Rough Set Theory , 2009, RSKT.

[21]  Yiyu Yao,et al.  Statistical Interpretations of Three-Way Decisions , 2015, RSKT.