GRIMMER (Granularity-Related Inconsistency of Means Mapped to Error Repeats) builds upon the GRIM test and allows for testing whether reported measures of variability are mathematically possible. GRIMMER relies upon the statistical phenomenon that variances display a simple repetitive pattern when the data is discrete, i.e. granular. This observation allows for the generation of an algorithm that can quickly identify whether a reported statistic of any size or precision is consistent with the stated sample size and granularity. My implementation of the test is available at PrePubMed and currently allows for testing variances, standard deviations, and standard errors for integer data. It is possible to extend the test to other measures of variability such as deviation from the mean, or apply the test to non-integer data such as data reported to halves or tenths. The ability of the test to identify inconsistent statistics relies upon four factors: (1) the sample size; (2) the granularity of the data; (3) the precision (number of decimals) of the reported statistic; and (4) the size of the standard deviation or standard error (but not the variance). The test is most powerful when the sample size is small, the granularity is large, the statistic is reported to a large number of decimal places, and the standard deviation or standard error is small (variance is immune to size considerations). This test has important implications for any field that routinely reports statistics for granular data to at least two decimal places because it can help identify errors in publications, and should be used by journals during their initial screen of new submissions. The errors detected can be the result of anything from something as innocent as a typo or rounding error to large statistical mistakes or unfortunately even fraud. In this report I describe the mathematical foundations of the GRIMMER test and the algorithm I use to implement it.
[1]
J. Ioannidis,et al.
Reproducibility in Science: Improving the Standard for Basic and Preclinical Research
,
2015,
Circulation research.
[2]
J. Leek,et al.
What Should Researchers Expect When They Replicate Studies? A Statistical View of Replicability in Psychological Science
,
2016,
Perspectives on psychological science : a journal of the Association for Psychological Science.
[3]
Jeffrey T Leek,et al.
An estimate of the science-wise false discovery rate and application to the top medical literature.
,
2014,
Biostatistics.
[4]
James A. J. Heathers,et al.
The GRIM Test
,
2017
.
[5]
Jelte M. Wicherts,et al.
The value of statistical tools to detect data fabrication
,
2016
.
[6]
G. Guyatt,et al.
Toward Fairness in Data Sharing.
,
2016,
The New England journal of medicine.
[7]
J. Ioannidis.
Why Most Published Research Findings Are False
,
2005,
PLoS medicine.
[8]
Farid Neema,et al.
Data sharing
,
1998
.