Bringing the Gold Standard into the Classroom: Replication in University Teaching

Reproducibility is held to be the gold standard for scientific research. The credibility of published work depends on being able to replicate the results. However, there are few incentives to conduct replication studies in political science. Replications are difficult to conduct, time-consuming, and hard to publish because of a presumed lack of originality. This article sees a solution in a profound change in graduate teaching. Universities should introduce replications as class assignments in methods training or invest in new stand-alone replication workshops to establish a culture of replication and reproducibility. This article will first discuss the benefits of conducting replications. The main part will focus on concrete steps in integrating replication in the classroom, from selecting a paper to final manuscripts. Drawing on the author's own teaching experience as well as that of others, particular emphasis will be on the pitfalls and challenges of letting students replicate work, as well as potential criticism. Only if universities nurture a reproducibility and replication culture can we ensure that the gold standard of reliable, credible, and valid research is not just an empty phrase.

[1]  P Rhodes,et al.  Publication. , 1983, Encyclopedic Dictionary of Archaeology.

[2]  M. Ridpath Free to Trade , 1995 .

[3]  James L. Gibson Cautious Reflections on a Data-Archiving Policy for Political Science , 1995 .

[4]  Paul S. Herrnson Replication, Verification, Secondary Analysis, and Data Collection in Political Science , 1995 .

[5]  Robert J-P. Hauck Oh Monsieur Pasteur, We Hardly Knew You! , 1995 .

[6]  W. Meyer Confirming, Infirming, and "Falsifying" Theories of Human Rights: Reflections on Smith, Bolyard, and Ippolito Through the Lens of Lakatos , 1999 .

[7]  D. Green,et al.  The Effects of Canvassing, Telephone Calls, and Direct Mail on Voter Turnout: A Field Experiment , 2000, American Political Science Review.

[8]  Helen V. Milner,et al.  Free to Trade: Democracies, Autocracies, and International Trade , 2000, American Political Science Review.

[9]  M. Peffley,et al.  A Multiple Values Model of Political Tolerance , 2001 .

[10]  M. Peffley,et al.  A Reply to Miller et al.: Replication Made Simple , 2001 .

[11]  Concept and Measurement Artifact in Multiple Values and Value Conflict Models , 2001 .

[12]  Edward D. Mansfield,et al.  Replication, Realism, and Robustness: Analyzing Political Regimes and International Trade , 2002, American Political Science Review.

[13]  Xinyuan Dai Political Regimes and International Trade: The Democratic Difference Revisited , 2002, American Political Science Review.

[14]  G. King,et al.  The Future of Replication , 2003 .

[15]  D. Green,et al.  Correction to Gerber and Green (2000), Replication of Disputed Findings, and Reply to Imai (2005) , 2005, American Political Science Review.

[16]  Kosuke Imai,et al.  Do Get-Out-the-Vote Calls Reduce Turnout? The Importance of Statistical Methods for Field Experiments , 2005, American Political Science Review.

[17]  Michael Price,et al.  To Replicate or Not To Replicate , 2011 .

[18]  G. Francis Publication bias and the failure of replication in experimental psychology , 2012, Psychonomic bulletin & review.

[19]  D. Lakens,et al.  Rewarding Replications , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[20]  Michael C. Frank,et al.  Teaching Replication , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[21]  T. Carsey Making DA-RT a Reality , 2013, PS: Political Science & Politics.

[22]  G. Alter,et al.  Data Access and Research Transparency in the Quantitative Tradition , 2013, PS: Political Science & Politics.

[23]  A. Lupia,et al.  Openness in Political Science: Data Access and Research Transparency , 2013, PS: Political Science & Politics.

[24]  Colin Elman,et al.  Data Access and Research Transparency in the Qualitative Tradition , 2013, PS: Political Science & Politics.

[25]  A. Moravcsik Transparency: The Revolution in Qualitative Research , 2013, PS: Political Science & Politics.

[26]  John Ishiyama,et al.  Replication, Research Transparency, and Journal Publications: Individualism, Community Models, and the Future of Replication Studies , 2013, PS: Political Science & Politics.

[27]  Mina Bissell,et al.  Reproducibility: The risks of the replication drive , 2013, Nature.

[28]  Alexia Katsanidou,et al.  Data Availability in Political Science Journals , 2013 .

[29]  Jan H. Höffler Teaching Replication in Quantitative Empirical Economics , 2014 .

[30]  A. Dafoe Science Deserves Better: The Imperative to Share Complete Replication Files , 2014 .

[31]  Martin Sill,et al.  Reproducible Research with R and RStudio. C. Gandrud (2013). Chapman & Hall/CRC: The R Series. 294 pages, ISBN‐13: 978‐1466572843. , 2014 .

[32]  Mark S. Bell,et al.  Questioning the Effect of Nuclear Weapons on Conflict , 2015 .