Leveraging Crowd for Collecting and Maintaining Educational Resources for Privacy Learning

As the Internet grows in importance, it is vital to develop methods and techniques for educating end-users to improve their awareness of online privacy. Web-based education tools have been proven effective in many domains and have been increasingly adopted by many online professional and educational services. However, the design and development of Web-based education tools for online privacy is still in its early stage. The traditional solutions always involve privacy experts who have sophisticated expertise. However, it is not clear how inspiring and effective these education tools are to general users of varying backgrounds. Furthermore, such involvement can make the tool development costly. In this paper, we discuss our experiences of designing a quiz-based privacy learning tool by leveraging the wisdom of the crowd on Amazon Mechanic Turk for two purposes: (1) designing the quiz questions for the tool; and (2) evaluating and assessing the effectiveness of the tool. Empirical study demonstrates that the crowd can provide high-quality educational materials for privacy learning.

[1]  Michael S. Bernstein,et al.  The future of crowd work , 2013, CSCW.

[2]  Z. Popovic,et al.  Increased Diels-Alderase activity through backbone remodeling guided by Foldit players , 2012, Nature Biotechnology.

[3]  Fred Paas,et al.  Usability evaluation of integrated e-learning , 2004 .

[4]  Santosh S. Vempala,et al.  Latent semantic indexing: a probabilistic analysis , 1998, PODS '98.

[5]  Gianluca Demartini,et al.  ZenCrowd: leveraging probabilistic reasoning and crowdsourcing techniques for large-scale entity linking , 2012, WWW.

[6]  Olivier Toubia,et al.  Improving Online Idea Generation Platforms and Customizing the Task Structure on the Basis of Consumers' Domain-Specific Knowledge , 2015 .

[7]  Yasuaki Sakamoto,et al.  Testing tournament selection in creative problem solving using crowds , 2011, ICIS.

[8]  Divesh Srivastava,et al.  Record linkage: similarity measures and algorithms , 2006, SIGMOD Conference.

[9]  Jeffrey V. Nickerson,et al.  Evaluating Design Solutions Using Crowds , 2011, AMCIS.

[10]  Sung Heum Lee,et al.  Usability Testing for Developing Effective Interactive Multimedia Software: Concepts, Dimensions, and Procedures , 1999, J. Educ. Technol. Soc..

[11]  Jie Ren,et al.  Increasing the crowd's capacity to create: how alternative generation affects the diversity, relevance and effectiveness of generated ads , 2014, Decis. Support Syst..

[12]  Jan Marco Leimeister,et al.  Leveraging Crowdsourcing: Activation-Supporting Components for IT-Based Ideas Competition , 2009, J. Manag. Inf. Syst..

[13]  Danah Boyd,et al.  Facebook privacy settings: Who cares? , 2010, First Monday.

[14]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[15]  Jan Marco Leimeister,et al.  The Effect of Rating Scales on Decision Quality and User Attitudes in Online Innovation Communities , 2012, Int. J. Electron. Commer..

[16]  Nir Ailon,et al.  Aggregation of Partial Rankings, p-Ratings and Top-m Lists , 2007, SODA '07.

[17]  Aniket Kittur,et al.  Searching for analogical ideas with crowds , 2014, CHI.

[18]  Martin Schreier,et al.  The Value of Crowdsourcing: Can Users Really Compete with Professionals in Generating New Product Ideas? , 2009 .

[19]  D. Critchlow.  Metric Methods for Analyzing Partially Ranked Data , 1986 .

[20]  Anders Albrechtslund,et al.  Online Social Networking as Participatory Surveillance , 2008, First Monday.

[21]  Marc Prensky,et al.  Digital game-based learning , 2000, CIE.

[22]  Yasuaki Sakamoto,et al.  Feature Selection in Crowd Creativity , 2011, HCI.

[23]  Daren C. Brabham.  MOVING THE CROWD AT THREADLESS , 2010 .

[24]  Jeffrey V. Nickerson,et al.  Cooks or cobblers?: crowd creativity through combination , 2011, CHI.

[25]  Elizabeth Gerber,et al.  Computer supported collective action , 2014, INTR.

[26]  Huaye Li,et al.  The Influence of Collective Opinion on True-False Judgment and Information-Sharing Decision , 2013, CogSci.

[27]  D. Kirkpatrick.  Techniques for evaluating training programs , 1979 .

[28]  Alessandro Bozzon,et al.  Answering search queries with CrowdSearcher , 2012, WWW.

[29]  Susan B. Barnes,et al.  A privacy paradox: Social networking in the United States , 2006, First Monday.

[30]  Monica Landoni,et al.  Evaluation of content activities in children's educational software , 2002 .

[31]  M. Trick,et al.  Voting schemes for which it can be difficult to tell who won the election , 1989 .

[32]  M. Kendall.  A NEW MEASURE OF RANK CORRELATION , 1938 .

[33]  Howard Chivers.  Personal Attributes and Privacy , 2005 .