Improving transparency and reproducibility through registration: The status of intervention trials published in clinical psychology journals.

OBJECTIVE Prospective registration increases the validity of randomized controlled trials (RCTs). In the United States, registration is a legal requirement for drugs and devices regulated by the Food and Drug Administration (FDA), and many biomedical journals refuse to publish trials that are not registered. Trials in clinical psychology have not been subject to these requirements; it is unknown to what extent they are registered. METHOD We searched the 25 highest-impact clinical psychology journals that published at least 1 RCT of a health-related psychological intervention in 2013. For included trials, we evaluated their registration status (prospective, retrospective, not registered) and the completeness of their outcome definitions. RESULTS We identified 163 articles that reported 165 RCTs; 73 (44%) RCTs were registered, of which only 25 (15%) were registered prospectively. Of registered RCTs, only 42 (58%) indicated their registration status in the publication. Only 2 (1% of all trials) were registered prospectively and defined their primary outcomes completely. For the primary outcome(s), 72 (99%) of all registrations defined the domain, 67 (92%) the time frame, and 48 (66%) the specific measurements. Only 19 (26%) and 5 (7%) defined the specific metric and method of aggregation, respectively, for all primary outcomes. CONCLUSIONS Very few reports of RCTs published in clinical psychology journals were registered prospectively and completely. Clinical psychology journals could improve transparency and reproducibility, as well as reduce bias, by requiring complete prospective trial registration for publication and by including trial registration numbers in all reports of RCTs. (PsycINFO Database Record

[1]  N. Lazar,et al.  The ASA Statement on p-Values: Context, Process, and Purpose , 2016 .

[2]  D. Kyriacou.  The Enduring Evolution of the P Value. , 2016, JAMA.

[3]  J. Ioannidis,et al.  Evolution of Reporting P Values in the Biomedical Literature, 1990-2015. , 2016, JAMA.

[4]  Vinay Prasad,et al.  Cancer Drugs Approved on the Basis of a Surrogate End Point and Subsequent Overall Survival: An Analysis of 5 Years of US Food and Drug Administration Approvals. , 2015, JAMA internal medicine.

[5]  B. Thombs,et al.  Transparency of Outcome Reporting and Trial Registration of Randomized Controlled Trials Published in the Journal of Consulting and Clinical Psychology , 2015, PloS one.

[6]  D. Korn,et al.  Clinical trial registration, reporting, publication and FDAAA compliance: a cross-sectional analysis and ranking of new drugs approved by the FDA in 2012 , 2015, BMJ Open.

[7]  D. Gorman.  'Everything works': the need to address confirmation bias in evaluations of drug misuse prevention interventions for adolescents. , 2015, Addiction.

[8]  S. Hollon,et al.  Does Publication Bias Inflate the Apparent Efficacy of Psychological Treatment for Major Depressive Disorder? A Systematic Review and Meta-Analysis of US National Institutes of Health-Funded Trials , 2015, PloS one.

[9]  C. Abraham,et al.  A checklist to improve reporting of group-based behaviour-change interventions , 2015, BMC Public Health.

[10]  Adrienne Stith Butler,et al.  Psychosocial Interventions for Mental and Substance Use Disorders: A Framework for Establishing Evidence-Based Standards , 2015 .

[11]  Michael C. Frank,et al.  Estimating the reproducibility of psychological science , 2015, Science.

[12]  R. Mulder,et al.  Is Mandatory Prospective Trial Registration Working to Prevent Publication of Unregistered Trials and Selective Outcome Reporting? An Observational Study of Five Psychiatry Journals That Mandate Prospective Clinical Trial Registration , 2015, PloS one.

[13]  R. Kaplan,et al.  Likelihood of Null Effects of Large NHLBI Clinical Trials Has Increased over Time , 2015, PloS one.

[14]  G. Gartlehner,et al.  Inadequate use and regulation of interventions against publication bias decreases their effectiveness: a systematic review , 2015, Journal of clinical epidemiology.

[15]  Brian A. Nosek,et al.  Promoting an open research culture , 2015, Science.

[16]  Á. Atallah,et al.  FDAAA legislation is working, but methodological flaws undermine the reliability of clinical trials: a cross-sectional study , 2015, PeerJ.

[17]  Thomas D. Cook,et al.  Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation , 2015, Prevention Science.

[18]  M. Kieny,et al.  Rationale for WHO's New Position Calling for Prompt Reporting and Public Disclosure of Interventional Clinical Trial Results , 2015, PLoS medicine.

[19]  Ben Goldacre,et al.  How to Get All Trials Reported: Audit, Better Data, and Individual Accountability , 2015, PLoS medicine.

[20]  J. Ioannidis,et al.  Call to improve transparency of trials of non-regulated interventions , 2015, BMJ : British Medical Journal.

[21]  A. O’Cathain,et al.  Process evaluation of complex interventions: Medical Research Council guidance , 2015, BMJ : British Medical Journal.

[22]  James Topping,et al.  Compliance with results reporting at ClinicalTrials.gov. , 2015, The New England journal of medicine.

[23]  B. Lo.  Sharing clinical trial data: maximizing benefits, minimizing risk. , 2015, JAMA.

[24]  E. Wager,et al.  Responsible Research Publication: International Standards for Authors , 2014, Prilozi.

[25]  Tianjing Li,et al.  Outcomes in Cochrane Systematic Reviews Addressing Four Common Eye Conditions: An Evaluation of Completeness and Comparability , 2014, PloS one.

[26]  Harvey J Motulsky,et al.  Common Misconceptions about Data Analysis and Statistics , 2014, The Journal of Pharmacology and Experimental Therapeutics.

[27]  P. Conrod,et al.  Europe Needs a Central, Transparent, and Evidence-Based Approval Process for Behavioural Prevention Interventions , 2014, PLoS medicine.

[28]  Daniel R Shanahan,et al.  Linked publications from a single trial: a thread of evidence , 2014, Trials.

[29]  Anton Kühberger,et al.  Publication Bias in Psychology: A Diagnosis Based on the Correlation between Effect Size and Sample Size , 2014, PloS one.

[30]  Alex O Holcombe,et al.  An Introduction to Registered Replication Reports at Perspectives on Psychological Science , 2014, Perspectives on psychological science : a journal of the Association for Psychological Science.

[31]  N. Pandis,et al.  Active implementation strategy of CONSORT adherence by a dental specialty journal improved randomized clinical trial reporting. , 2014, Journal of clinical epidemiology.

[32]  R. Saitz.  The best evidence for alcohol screening and brief intervention in primary care supports efficacy, at best, not effectiveness: You say tomāto, I say tomăto? That’s not all it’s about , 2014, Addiction Science & Clinical Practice.

[33]  N. Božina,et al.  Prevalence of genetic polymorphisms of CYP2C9 and VKORC1 — Implications for warfarin management and outcome in Croatian patients with acute stroke , 2014, Journal of the Neurological Sciences.

[34]  Brian A. Nosek,et al.  Publication and other reporting biases in cognitive sciences: detection, prevalence, and prevention , 2014, Trends in Cognitive Sciences.

[35]  David Moher,et al.  Reducing waste from incomplete or unusable reports of biomedical research , 2014, The Lancet.

[36]  Harlan M Krumholz,et al.  Increasing value and reducing waste: addressing inaccessible research , 2014, The Lancet.

[37]  Michèle B. Nuijten,et al.  Why Publishing Everything Is More Effective than Selective Publishing of Statistically Significant Results , 2014, PloS one.

[38]  U. Dirnagl,et al.  Biomedical research: increasing value, reducing waste , 2014, The Lancet.

[39]  Roderik F. Viergever,et al.  The Quality of Registration of Clinical Trials: Still a Problem , 2014, PloS one.

[40]  E. Eich.  Business Not as Usual , 2014, Psychological science.

[41]  Leif D. Nelson,et al.  P-Curve: A Key to the File Drawer , 2013, Journal of experimental psychology. General.

[42]  Jung-Hye Kwon,et al.  The efficacy of imagery rescripting (IR) for social phobia: a randomized controlled trial. , 2013, Journal of behavior therapy and experimental psychiatry.

[43]  J. Valentine,et al.  Outcome-Reporting Bias in Education Research , 2013 .

[44]  J. Meerpohl,et al.  How Psychiatry Journals Support the Unbiased Translation of Clinical Research. A Cross-Sectional Study of Editorial Policies , 2013, PloS one.

[45]  J. Piacentini,et al.  Integrated Behavior Therapy for Selective Mutism: a randomized controlled pilot study. , 2013, Behaviour research and therapy.

[46]  E. Mayo-Wilson,et al.  Protocol for CONSORT-SPI: an extension for social and psychological interventions , 2013, Implementation Science.

[47]  Paul Montgomery,et al.  The Oxford Implementation Index: a new tool for incorporating implementation data into systematic reviews and meta-analyses. , 2013, Journal of clinical epidemiology.

[48]  C. Gamble,et al.  Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias — An Updated Review , 2013, PloS one.

[49]  E. Mayo-Wilson,et al.  Developing a reporting guideline for social and psychological intervention trials , 2013, Trials.

[50]  E. Mayo-Wilson,et al.  Reporting Quality of Social and Psychological Intervention Trials: A Systematic Review of Reporting Guidelines and Trial Publications , 2013, PloS one.

[51]  E. Stice,et al.  Effectiveness of peer-led dissonance-based eating disorder prevention groups: results from two randomized pilot trials. , 2013, Behaviour research and therapy.

[52]  Joel Lexchin,et al.  Surrogate outcomes in clinical trials: a cautionary tale. , 2013, JAMA internal medicine.

[53]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[54]  N. Schneiderman,et al.  Gender-specific effects of an augmented written emotional disclosure intervention on posttraumatic, depressive, and HIV-disease-related outcomes: a randomized, controlled trial. , 2013, Journal of consulting and clinical psychology.

[55]  D. Rennie,et al.  SPIRIT 2013 statement: defining standard protocol items for clinical trials. , 2013, Annals of internal medicine.

[56]  J. Musch,et al.  Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for depression. , 2013, Journal of consulting and clinical psychology.

[57]  J. Wicherts,et al.  The Rules of the Game Called Psychological Science , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[58]  J. Ioannidis.  Why Science Is Not Necessarily Self-Correcting , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[59]  Han L. J. van der Maas,et al.  Science Perspectives on Psychological an Agenda for Purely Confirmatory Research on Behalf Of: Association for Psychological Science , 2022 .

[60]  E. Masicampo,et al.  A peculiar prevalence of p values just below .05 , 2012, Quarterly journal of experimental psychology.

[61]  J. Musch,et al.  Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for schizophrenia , 2012, Schizophrenia Research.

[62]  Brian A. Nosek,et al.  Scientific Utopia , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[63]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[64]  Daniele Fanelli,et al.  Negative results are disappearing from most disciplines and countries , 2011, Scientometrics.

[65]  Nicholas C. Ide,et al.  The ClinicalTrials.gov results database--update and key issues. , 2011, The New England journal of medicine.

[66]  B. Thombs,et al.  Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: A systematic review. , 2011, Journal of psychosomatic research.

[67]  R. Brand,et al.  Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial. , 2010, Archives of internal medicine.

[68]  D. Moher,et al.  Transparent and accurate reporting increases reliability, utility, and impact of your research: reporting guidelines and the EQUATOR Network , 2010, BMC medicine.

[69]  D. Moher,et al.  CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials , 2010, Journal of clinical epidemiology.

[70]  Gerhard Andersson,et al.  Efficacy of cognitive–behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias , 2010, British Journal of Psychiatry.

[71]  F. Song,et al.  Dissemination and publication of research findings: an updated review of related biases. , 2010, Health technology assessment.

[72]  David Moher,et al.  Comparison of registered and published primary outcomes in randomized controlled trials. , 2009, JAMA.

[73]  G. Andersson,et al.  The effects of psychotherapy for adult depression are overestimated: a meta-analysis of study quality and effect size , 2009, Psychological Medicine.

[74]  J. Ioannidis.  Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[75]  G. Staines,et al.  Bias in Meta-Analytic Estimates of the Absolute Efficacy of Psychotherapy , 2007 .

[76]  E. von Elm,et al.  Full publication of results initially presented in abstracts. , 2007, The Cochrane database of systematic reviews.

[77]  Alla Keselman,et al.  Registering a clinical trial in ClinicalTrials.gov. , 2007, Chest.

[78]  D. Ames.  Registration of clinical trials submitted for publication in International Psychogeriatrics , 2006, International Psychogeriatrics.

[79]  J. Ioannidis.  Why Most Published Research Findings Are False , 2005 .

[80]  John Hoey,et al.  Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors , 2005, Canadian Medical Association Journal.

[81]  D. Altman,et al.  Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors , 2005, BMJ : British Medical Journal.

[82]  V. McCormack,et al.  Issues in the reporting of epidemiological studies: a survey of recent practice , 2004, BMJ : British Medical Journal.

[83]  D. Altman,et al.  Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research , 2004, Canadian Medical Association Journal.

[84]  R. Horton,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors , 2004, Canadian Medical Association Journal.

[85]  A. Hrõbjartsson,et al.  Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. , 2004, JAMA.

[86]  Gerd Antes,et al.  Under-reporting of clinical trials is unethical , 2003, The Lancet.

[87]  D. Rennie,et al.  Publication bias in editorial decision making. , 2002, JAMA.

[88]  Steve McDonald,et al.  Development of the Cochrane Collaboration’s Central Register of Controlled Clinical Trials , 2002, Evaluation & the health professions.

[89]  H. Cooper,et al.  Finding the Missing Science: The Fate of Studies Submitted for Review by a Human Subjects Committee , 1997 .

[90]  I. Olkin,et al.  Improving the quality of reporting of randomized controlled trials. The CONSORT statement. , 1996, JAMA.

[91]  K. Dickersin,et al.  Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. , 1992, JAMA.

[92]  I Chalmers,et al.  Underreporting research is scientific misconduct. , 1990, JAMA.

[93]  C. Meinert,et al.  Toward prospective registration of clinical trials. , 1988, IRB.

[94]  R. Simes.  Publication bias: the case for an international registry of clinical trials. , 1986, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[95]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .