P-Hacking in Orthopaedic Literature: A Twist to the Tail.

BACKGROUND "P-hacking" occurs when researchers preferentially select data or statistical analyses until nonsignificant results become significant. We wanted to evaluate if the phenomenon of p-hacking was evident in orthopaedic literature. METHODS We text-mined through all articles published in three top orthopaedic journals in 2015. For anonymity, we cipher-coded the three journals. We included all studies that reported a single p value to answer their main hypothesis. These p values were then charted and frequency graphs were generated to illustrate any evidence of p-hacking. Binomial tests were employed to look for evidence of evidential value and significance of p-hacking. RESULTS Frequency plots for all three journals revealed evidence of p-hacking. Binomial tests for all three journals were significant for evidence of evidential value (p < 0.0001 for all). However, the binomial test for p-hacking was significant only for one journal (p = 0.0092). CONCLUSIONS P-hacking is an evolving phenomenon that threatens to jeopardize the evidence-based practice of medicine. Although our results show that there is good evidential value for orthopaedic literature published in our top journals, there is some evidence of p-hacking of which authors and readers should be wary.

[1]  A J Sutton,et al.  Publication and related biases. , 2000, Health technology assessment.

[2]  Daniele Fanelli,et al.  Negative results are disappearing from most disciplines and countries , 2011, Scientometrics.

[3]  Alan S. Gerber,et al.  Publication Bias in Empirical Sociological Research , 2008 .

[4]  David B. Allison,et al.  Inappropriate Fiddling with Statistical Analyses to Obtain a Desirable P-value: Tests to Detect its Presence in Published Literature , 2012, PloS one.

[5]  J. Ioannidis Why Most Published Research Findings Are False , 2019, CHANCE.

[6]  R. Lanfear,et al.  The Extent and Consequences of P-Hacking in Science , 2015, PLoS biology.

[7]  Leif D. Nelson,et al.  P-Curve: A Key to the File Drawer , 2013, Journal of experimental psychology. General.

[8]  Leif D. Nelson,et al.  False-Positive Psychology , 2011, Psychological science.

[9]  Regina Nuzzo,et al.  Scientific method: Statistical errors , 2014, Nature.

[10]  S. Golder,et al.  The effectiveness and cost-effectiveness of prophylactic removal of wisdom teeth. , 2000, Health technology assessment.

[11]  M. Bhandari,et al.  Publication bias in orthopaedic research: an analysis of scientific factors associated with publication in the Journal of Bone and Joint Surgery (American Volume). , 2008, The Journal of bone and joint surgery. American volume.

[12]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[13]  C. Begg,et al.  Publication bias : a problem in interpreting medical data , 1988 .