Are units of analysis properly considered in orthodontic meta-analyses?

BACKGROUND Assessment of unit of analysis (UoA) in meta-analysis with cluster, split-mouth, repeated measures designs, and multiple intervention groups is a fundamental step in the analysis. The objective of this study was to evaluate the UoAs in orthodontic meta-analysis and determine the error of the analysis. METHODS An electronic search was conducted in the Cochrane Library and PubMed to identify orthodontic systematic reviews (SRs) with meta-analyses published in Cochrane and in the highest impact orthodontic journals between 1 January 2013 and 31 December 2022. SRs with meta-analysis assessing at least one of the following UoAs; cluster trials, crossover trials, multiple observations, or multiple intervention groups were included. Screening and data extraction were undertaken by two investigators independently. Descriptive statistics for the study characteristics were provided. The associations between avoiding the unit analysis error (yes, no) and the study characteristics were examined using Fisher's exact test and chi-square test. Logistic regression was undertaken for the significant predictors. RESULTS Eighty SRs were deemed eligible for inclusion. Only 30 per cent of the included SRs avoided UoA errors. Compared to the split-mouth design, repeated measures designs had higher odds of avoiding UoA error (odds ratio: 9.6, 95% confidence interval: 2.8-32.3, P < 0.001). In contrast, fewer odds of avoiding the UoA error were found in the cluster design (OR: 0.2, 95% CI: 0.4-1.3, P = 0.09). Though multiple intervention groups have higher odds of avoiding UoA error than split-mouth studies, this was not statistically significant (OR: 2.1, 95% CI: 0.5-8, P = 0.28). None of the SRs characteristics have influenced the appropriate handling of the unit analysis except the type of the journal; the odds of avoiding the UoA error were higher in Cochrane reviews than the non-Cochrane reviews (OR: 3.3, 95% CI: 1.2-8.7, P = 0.02), and the number of authors (P < 0.05). CONCLUSIONS UoA errors are common in orthodontic meta-analyses and were only partially avoided in split-mouth design, repeated measures design, and multiple intervention groups.

[1]  N. Pandis,et al.  Meta-analysis of split-mouth studies. , 2023, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[2]  N. Pandis,et al.  Meta-analysis of cluster-randomized trials. , 2023, American journal of orthodontics and dentofacial orthopedics : official publication of the American Association of Orthodontists, its constituent societies, and the American Board of Orthodontics.

[3]  N. Pandis,et al.  Do longitudinal orthodontic trials use appropriate statistical analyses? A meta-epidemiological study. , 2021, European journal of orthodontics.

[4]  P. Cuijpers,et al.  Doing Meta-Analysis with R , 2021 .

[5]  H. Worthington,et al.  The reporting quality of split-mouth trials in orthodontics according to CONSORT guidelines: 2015-19. , 2021, European journal of orthodontics.

[6]  J. Higgins,et al.  Including variants on randomized trials , 2019, Cochrane Handbook for Systematic Reviews of Interventions.

[7]  J. Higgins,et al.  Choosing effect measures and computing estimates of effect , 2019, Cochrane Handbook for Systematic Reviews of Interventions.

[8]  Spyridon N Papageorgiou,et al.  Registration in the international prospective register of systematic reviews (PROSPERO) of systematic review protocols was associated with increased review quality. , 2018, Journal of clinical epidemiology.

[9]  Zhen Wang,et al.  Guidelines for reporting meta-epidemiological methodology research , 2017, Evidence-Based Medicine.

[10]  P. Cuijpers,et al.  Pre-post effect sizes should be avoided in meta-analyses , 2016, Epidemiology and Psychiatric Sciences.

[11]  Mike W-L Cheung,et al.  Modeling dependent effect sizes with three-level meta-analyses: a structural equation modeling approach. , 2014, Psychological methods.

[12]  Ewout W. Steyerberg,et al.  Individual participant data meta-analyses should not ignore clustering , 2013, Journal of clinical epidemiology.

[13]  N. Pandis,et al.  Are clustering effects accounted for in statistical analysis in leading dental specialty journals? , 2013, Journal of dentistry.

[14]  N. Pandis,et al.  Does published orthodontic research account for clustering effects during statistical data analysis? , 2012, European journal of orthodontics.

[15]  E. Lesaffre,et al.  The design and analysis of split‐mouth studies: What statisticians and clinicians should know , 2009, Statistics in medicine.

[16]  A. Whitehead,et al.  Meta-analysis of individual patient data versus aggregate data from longitudinal clinical trials , 2009, Clinical trials.

[17]  Jaime L Peters,et al.  Meta-analysis of repeated measures study designs. , 2008, Journal of evaluation in clinical practice.

[18]  Douglas G. Altman,et al.  Statistical Methods for Examining Heterogeneity and Combining Results from Several Studies in Meta‐Analysis , 2008 .

[19]  Douglas G Altman,et al.  Epidemiology and reporting of randomised trials published in PubMed journals , 2005, The Lancet.

[20]  J. Grimshaw,et al.  Unit of analysis errors should be clarified in meta-analyses , 2003, BMJ : British Medical Journal.

[21]  A Donner,et al.  Statistical methods for the meta-analysis of cluster randomization trials , 2001, Statistical methods in medical research.

[22]  A. Scott,et al.  A simple method for the analysis of clustered binary data. , 1992, Biometrics.