Gathering and evaluating innovation ideas using crowdsourcing: Impact of the idea title and the description on the number of votes in each phase of a two‐phase crowdsourcing project

Organizations are using crowdsourcing to capture innovation knowledge from the crowd in the form of ideas and then using the crowd to evaluate those ideas using votes. In this paper, we investigate a crowdsourcing setting in which Canada solicited information from its citizens to develop a digital transformation strategy. Canada used a two‐phase approach. Phase 1 was used to determine which ideas had the largest number of crowd votes, whereas in Phase 2, the crowd voted on the 30 leading vote‐getting ideas to determine the three winning ideas.

[1]  Kristina Lerman,et al.  The myopia of crowds: Cognitive load and collective evaluation of answers on Stack Exchange , 2016, PloS one.

[2]  Jan Marco Leimeister,et al.  Revealing the Impact of the Crowdsourcing Experience on the Engagement Process , 2016, ICIS.

[3]  Daniel E. O'Leary,et al.  On the relationship between number of votes and sentiment in crowdsourcing ideas and comments for innovation: A case study of Canada's digital compass , 2016, Decis. Support Syst..

[4]  Margaret R. Garnsey,et al.  Natural Language Processing in Accounting, Auditing and Finance: A Synthesis of the Literature with a Roadmap for Future Research , 2016, Intell. Syst. Account. Finance Manag..

[5]  Özlem Uzuner,et al.  Do Sentiments Matter in Fraud Detection? Estimating Semantic Orientation of Annual Reports , 2016, Intell. Syst. Account. Finance Manag..

[6]  Daniel E. O'Leary,et al.  Twitter Mining for Discovery, Prediction and Causality: Applications and Methodologies , 2015, Intell. Syst. Account. Finance Manag..

[7]  O DANIELE.,et al.  TWITTER MINING FOR DISCOVERY , PREDICTION AND CAUSALITY : APPLICATIONS AND METHODOLOGIES , 2015 .

[8]  Michael Vitale,et al.  The Wisdom of Crowds , 2015, Cell.

[9]  Paul Wright.  Protecting Oracle Database 12c , 2014, Apress.

[10]  Mausam,et al.  Crowdsourcing Multi-Label Classification for Taxonomy Creation , 2013, HCOMP.

[11]  Matt Post,et al.  Constructing Parallel Corpora for Six Indian Languages via Crowdsourcing , 2012, WMT@NAACL-HLT.

[12]  Daniel E. O'Leary,et al.  Blog mining-review and extensions: "From each according to his opinion" , 2011, Decis. Support Syst..

[13]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[14]  R. Ramakrishnan,et al.  Crowdsourcing systems on the World-Wide Web , 2011, Commun. ACM.

[15]  Martin Schader,et al.  Managing the Crowd: Towards a Taxonomy of Crowdsourcing Processes , 2011, AMCIS.

[16]  Tim Loughran,et al.  When is a Liability not a Liability? Textual Analysis, Dictionaries, and 10-Ks , 2010 .

[17]  J. Pennebaker,et al.  The Psychological Meaning of Words: LIWC and Computerized Text Analysis Methods , 2010 .

[18]  Katja Hutter,et al.  Evaluation Games - How to Make the Crowd your Jury , 2010, GI Jahrestagung.

[19]  Paul Michael Di Gangi,et al.  Steal my idea! Organizational adoption of user innovations from a user innovation community: A case study of Dell IdeaStorm , 2009, Decis. Support Syst..

[20]  S. West,et al.  The Analysis of Count Data: A Gentle Introduction to Poisson Regression and Its Alternatives , 2009, Journal of personality assessment.

[21]  Brendan T. O'Connor,et al.  Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks , 2008, EMNLP.

[22]  D. MacKenzie,et al.  The use of knowledge about society , 2008 .

[23]  Daren C. Brabham.  Crowdsourcing as a Model for Problem Solving , 2008 .

[24]  O. Bjelland,et al.  An Inside View of IBM's 'Innovation Jam' , 2008 .

[25]  Yan Liu,et al.  Looking for great ideas: analyzing the innovation jam , 2007, WebKDD/SNA-KDD '07.

[26]  R. Jackson,et al.  Ethnic difference in the relationship between acute inflammation and serum ferritin in US adult males , 2007, Epidemiology and Infection.

[27]  Philip J. Stone,et al.  The general inquirer: A computer system for content analysis and retrieval based on the sentence as a unit of information , 2007 .

[28]  Daniel E. O'Leary,et al.  Knowledge Acquisition From Multiple Experts: An Empirical Study , 1998 .

[29]  Daniel E. O'Leary,et al.  Determining Differences in Expert Judgment: Implications for Knowledge Acquisition and Validation* , 1993 .

[30]  Paul Gray,et al.  Group decision support systems , 1987, Decis. Support Syst..

[31]  John C. Cherniavsky,et al.  Validation, Verification, and Testing of Computer Software , 1982, CSUR.

[32]  Philip J. Stone,et al.  A computer approach to content analysis: studies using the General Inquirer system , 1963, AFIPS Spring Joint Computing Conference.