Introduction to the Special Section on Advancing Our Methods and Practices

Psychological science is in the midst of a sea change. Over the last few years, our field’s confidence in the status quo has been shaken by a number of largely unrelated events that happened to coincide—Jonah Lehrer’s (2010) widely read New Yorker article on the effects of publication bias in science, Bem’s (2011) controversial paper on precognition, a rising concern about direct replication, the Stapel fraud case (Tilburg University, 2011), and the publication of several troubling critiques of current practices in research and publishing (e.g., Simmons, Nelson, & Simonsohn, 2011; Vul, Harris, Winkielman, & Pashler, 2009). Although the ensuing crisis of confidence was by no means the first that psychology has faced (see e.g., Rosenthal & Rubin, 1978; Sears, 1986; Wicker, 1969), this one seems to have resonated more widely and more deeply. The convergence of events within our field was situated within a broader context of similar issues emerging across a range of scientific disciplines, from cancer research to genetics to neuroscience (e.g., Begley & Ellis, 2012; Button et al., 2013; Fanelli, 2012; Ioannidis, 2005; Moonesinghe, Khoury, & Janssens, 2007). Meanwhile, online communication, media attention, and a series of conference symposia and journal issues kept the critiques and concerns front and center. The first wave of responses to the sense of crisis understandably focused on problems—many of which had been raised before and even repeatedly (Cohen, 1992; Greenwald, 1975; Maxwell, 2004; Rosenthal, 1979)—but which in this new context seemed more urgently and insistently to demand the field’s consideration. A chorus of critiques focused our attention on issues of publication bias, underpowered studies, replication, flashy findings, and questionable research practices (e.g., Bakker, van Dijk, & Wicherts, 2012; John, Loewenstein, & Prelec, 2012; Ledgerwood & Sherman, 2012; Nosek, Spies, & Motyl, 2012; Pashler & Wagenmakers, 2012). Some embraced these critiques wholeheartedly, whereas others pushed back, arguing that some of the problems were overstated or oversimplified. This first wave of responses was loud enough and big enough to overcome the inevitable inertia of an existing system and propel the field into forward motion. We can and surely should debate which problems are most pressing and which solutions most suitable (e.g., Cesario, 2014; Fiedler, Kutzner, & Krueger, 2012; Murayama, Pekrun, & Fiedler, 2013; Stroebe & Strack, 2014). But at this point, most can agree that there are some real problems with the status quo. Many researchers feel poised to change their current practices in an effort to improve our science. Already, new initiatives and journal policies have started moving the field forward to meet some of the recently articulated challenges head on (Chambers & Munafo, 2013; Eich, 2014; LeBel et al., 2013; Open Science Framework, 2014; PSPB, 2014; Spellman, 2013). It is in many ways an exciting time: Our momentum has placed psychological science at the forefront of a broader movement to improve standards and practices across scientific disciplines (see e.g., McNutt, 2014). But of course, change also involves uncertainty. For the average researcher or student standing on the shifting sands of new journal policies, conflicting reviewer standards, and ongoing debates about best practices, the view can seem rather turbulent. One might reasonably wonder, “What should I be doing differently in my own research? Do I really need to triple all my sample sizes? Is it ever okay to peek at my data? What should I conclude when I run the same study twice and get different results? And in the midst of all of this, how should I adapt my own expectations as a reviewer or editor . . . and what can I expect from others reviewing my manuscripts?” This special section brings together a collection of articles that address exactly these kinds of questions. The goal is to provide a concrete set of practical best practices—that is, things we can change right now about the way we conduct and evaluate research that will make our science better. The section opens with an overview of cutting-edge tools that enable researchers to increase the evidential value of their studies (Lakens & Evers, 2014, 529448 PPSXXX10.1177/1745691614529448LedgerwoodIntroduction to the Special Section research-article2014