Unit of analysis issues in laboratory-based research

Many studies in the biomedical research literature report analyses that fail to recognise important data dependencies from multilevel or complex experimental designs. Statistical inferences resulting from such analyses are unlikely to be valid and are often potentially highly misleading. Failure to recognise this as a problem is often referred to in the statistical literature as a unit of analysis (UoA) issue. Here, by analysing two example datasets in a simulation study, we demonstrate the impact of UoA issues on study efficiency and estimation bias, and highlight where errors in analysis can occur. We also provide code (written in R) as a resource to help researchers undertake their own statistical analyses.

[1]  D. Bates,et al.  Mixed-Effects Models in S and S-PLUS , 2001 .

[2]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[3]  Stanley E Lazic,et al.  The problem of pseudoreplication in neuroscientific studies: is it affecting your analysis? , 2010, BMC Neuroscience.

[4]  P. Diggle Analysis of Longitudinal Data , 1995 .

[5]  Kurt Hornik,et al.  The Comprehensive R Archive Network , 2012 .

[6]  Yvonne Vergouwe,et al.  Prediction models for clustered data: comparison of a random intercept and standard regression model , 2013, BMC Medical Research Methodology.

[7]  John P. A. Ioannidis,et al.  Research: increasing value, reducing waste 2 , 2014 .

[8]  R. Tibshirani,et al.  Increasing value and reducing waste in research design, conduct, and analysis , 2014, The Lancet.

[9]  Linda M. Frazier,et al.  The unit of analysis error in studies about physicians’ patient care behavior , 1992, Journal of General Internal Medicine.

[10]  Matthijs Verhage,et al.  A solution to dependency: using multilevel analysis to accommodate nested data , 2014, Nature Neuroscience.

[11]  N. Pandis,et al.  Are clustering effects accounted for in statistical analysis in leading dental specialty journals? , 2013, Journal of dentistry.

[12]  R. Fisher XV.—The Correlation between Relatives on the Supposition of Mendelian Inheritance. , 1919, Transactions of the Royal Society of Edinburgh.

[13]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[14]  P. Bacchetti,et al.  Sample size calculations in clinical research. , 2002, Anesthesiology.

[15]  S. Bustin,et al.  Improving the reliability of peer-reviewed publications: We are all in it together , 2015, Biomolecular detection and quantification.

[16]  Douglas G Altman,et al.  Statistics Notes: Units of analysis , 1997, BMJ.

[17]  F. Mair,et al.  Qualitative systematic reviews of treatment burden in stroke, heart failure and diabetes - Methodological challenges and solutions , 2013, BMC Medical Research Methodology.

[18]  Gordon H Guyatt,et al.  Addressing the Unit of Analysis in Medical Care Studies: A Systematic Review , 2008, Medical care.

[19]  I. Cuthill,et al.  Survey of the Quality of Experimental Design, Statistical Analysis and Reporting of Research Using Animals , 2009, PloS one.

[20]  D. Moher,et al.  The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration , 2001, Annals of Internal Medicine.

[21]  Douglas G Altman,et al.  The time has come to register diagnostic and prognostic research. , 2014, Clinical chemistry.

[22]  Peter Green,et al.  SIMR: an R package for power analysis of generalized linear mixed models by simulation , 2016 .

[23]  Rodney X. Sturdivant,et al.  Applied Logistic Regression: Hosmer/Applied Logistic Regression , 2005 .

[24]  Journals unite for reproducibility , 2014, Nature.

[25]  Karla Hemming,et al.  Sample size calculations for cluster randomised controlled trials with a fixed number of clusters , 2011, BMC medical research methodology.

[26]  Ken Aho,et al.  Foundational and Applied Statistics for Biologists Using R , 2013 .

[27]  Roel Bosker,et al.  Multilevel analysis : an introduction to basic and advanced multilevel modeling , 1999 .

[28]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[29]  Steven G. Gilmour,et al.  Statistical Principles for the Design of Experiments: Fractional replication , 2012 .

[30]  D. Bates,et al.  Fitting Linear Mixed-Effects Models Using lme4 , 2014, 1406.5823.

[31]  Pie Müller,et al.  Power analysis for generalized linear mixed models in ecology and evolution , 2014, Methods in ecology and evolution.

[32]  Sanford Weisberg,et al.  An R Companion to Applied Regression , 2010 .

[33]  Joseph Hilbe,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[34]  I. Cuthill,et al.  Reporting : The ARRIVE Guidelines for Reporting Animal Research , 2010 .

[35]  Helen Brown,et al.  Applied Mixed Models in Medicine , 2000, Technometrics.

[36]  Ross Ihaka,et al.  Gentleman R: R: A language for data analysis and graphics , 1996 .

[37]  Charles E. Heckler,et al.  Introduction to Mixed Modelling. Beyond Regression and Analysis of Variance , 2008, Technometrics.

[38]  N. Freemantle,et al.  Ophthalmic statistics note 1: unit of analysis , 2013, British Journal of Ophthalmology.

[39]  James M. Eales,et al.  RIPOSTE: a framework for improving the design and analysis of laboratory-based research , 2015, eLife.