The use and misuse of statistical methodologies in pharmacology research.

Descriptive, exploratory, and inferential statistics are necessary components of hypothesis-driven biomedical research. Despite the ubiquitous need for these tools, the emphasis on statistical methods in pharmacology has become dominated by inferential methods often chosen more by the availability of user-friendly software than by any understanding of the data set or the critical assumptions of the statistical tests. Such frank misuse of statistical methodology and the quest to reach the mystical α<0.05 criteria has hampered research via the publication of incorrect analysis driven by rudimentary statistical training. Perhaps more critically, a poor understanding of statistical tools limits the conclusions that may be drawn from a study by divorcing the investigator from their own data. The net result is a decrease in quality and confidence in research findings, fueling recent controversies over the reproducibility of high profile findings and effects that appear to diminish over time. The recent development of "omics" approaches leading to the production of massive higher dimensional data sets has amplified these issues making it clear that new approaches are needed to appropriately and effectively mine this type of data. Unfortunately, statistical education in the field has not kept pace. This commentary provides a foundation for an intuitive understanding of statistics that fosters an exploratory approach and an appreciation for the assumptions of various statistical tests that hopefully will increase the correct use of statistics, the application of exploratory data analysis, and the use of statistical study design, with the goal of increasing reproducibility and confidence in the literature.

[1]  Walter L. Smith Probability and Statistics , 1959, Nature.

[2]  J. Ioannidis,et al.  Persistence of contradicted claims in the literature. , 2007, JAMA.

[3]  F. Prinz,et al.  Believe it or not: how much can we rely on published data on potential drug targets? , 2011, Nature Reviews Drug Discovery.

[4]  Jacob Cohen The earth is round (p < .05) , 1994 .

[5]  S. Geisser,et al.  On methods in the analysis of profile data , 1959 .

[6]  F. E. Grubbs Procedures for Detecting Outlying Observations in Samples , 1969 .

[7]  Roger E. Kirk,et al.  Statistics: An Introduction , 1998 .

[8]  M. Kendall Statistical Methods for Research Workers , 1937, Nature.

[9]  J. Crabbe,et al.  Genetics of mouse behavior: interactions with laboratory environment. , 1999, Science.

[10]  M. Lyon An Interview With... , 2004, Nature Reviews Genetics.

[11]  David S. Moore,et al.  Undergraduate Programs and the Future of Academic Statistics , 2001 .

[12]  Benjamin Peirce,et al.  Criterion for the rejection of doubtful observations , 1852 .

[13]  C. Dunnett A Multiple Comparison Procedure for Comparing Several Treatments with a Control , 1955 .

[14]  W. W. Daniel,et al.  Applied Nonparametric Statistics , 1978 .

[15]  Franz H Messerli,et al.  Chocolate consumption, cognitive function, and Nobel laureates. , 2012, The New England journal of medicine.

[16]  M. Friedman The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[17]  Sydney Brenner,et al.  An interview with... Sydney Brenner. Interview by Errol C. Friedberg. , 2008, Nature reviews. Molecular cell biology.

[18]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[19]  M. H. Ensom,et al.  Post Hoc Power Analysis: An Idea Whose Time Has Passed? , 2001, Pharmacotherapy.

[20]  W. Kruskal,et al.  Use of Ranks in One-Criterion Variance Analysis , 1952 .

[21]  Clyde Young Kramer,et al.  Extension of multiple range tests to group means with unequal numbers of replications , 1956 .

[22]  G. Gigerenzer,et al.  Teaching Bayesian reasoning in less than two hours. , 2001, Journal of experimental psychology. General.

[23]  F. J. Anscombe,et al.  Graphs in Statistical Analysis , 1973 .

[24]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[25]  Mohini P. Barde,et al.  What to use to express the variability of data: Standard deviation or standard error of mean? , 2012, Perspectives in clinical research.

[26]  J. Rodgers,et al.  Thirteen ways to look at the correlation coefficient , 1988 .

[27]  M. Fay,et al.  Wilcoxon-Mann-Whitney or t-test? On assumptions for hypothesis tests and multiple interpretations of decision rules. , 2010, Statistics surveys.

[28]  D. Heisey,et al.  The Abuse of Power , 2001 .

[29]  F. Wilcoxon Individual Comparisons by Ranking Methods , 1945 .

[30]  C. Spearman The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[31]  S. Potkin,et al.  What Is Causing the Reduced Drug-Placebo Difference in Recent Schizophrenia Clinical Trials and What Can be Done About It? , 2008, Schizophrenia bulletin.

[32]  C. Begley,et al.  Drug development: Raise standards for preclinical cancer research , 2012, Nature.

[33]  H. Huynh,et al.  Estimation of the Box Correction for Degrees of Freedom from Sample Data in Randomized Block and Split-Plot Designs , 1976 .

[34]  Edward H Livingston,et al.  Who was student and why do we care so much about his t-test? , 2004, The Journal of surgical research.

[35]  B. L. Welch The generalisation of student's problems when several different population variances are involved. , 1947, Biometrika.

[36]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[37]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[38]  J. Ioannidis Why Most Discovered True Associations Are Inflated , 2008, Epidemiology.

[39]  W. Dixon,et al.  Simplified Statistics for Small Numbers of Observations , 1951 .

[40]  Len Thomas,et al.  Retrospective Power Analysis , 1997 .

[41]  J. Brooks Why most published research findings are false: Ioannidis JP, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, Ioannina, Greece , 2008 .

[42]  T. Perneger What's wrong with Bonferroni adjustments , 1998, BMJ.

[43]  John W. Tukey,et al.  Exploratory Data Analysis. , 1979 .

[44]  Errol C. Friedberg,et al.  Sydney Brenner , 2008, Nature Reviews Molecular Cell Biology.

[45]  Jacob Cohen Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[46]  Jonah Lehrer The Truth Wears Off , 2011 .

[47]  J. Ioannidis Why Most Published Research Findings Are False , 2005, PLoS medicine.

[48]  Alexander R. Pico,et al.  Finding the Right Questions: Exploratory Pathway Analysis to Enhance Biological Discovery in Large Datasets , 2010, PLoS biology.

[49]  Welch Bl THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[50]  J. Ioannidis Contradicted and Initially Stronger Effects in Highly Cited Clinical Research , 2005 .