Improved Statistical Modeling of Tumor Growth and Treatment Effect in Preclinical Animal Studies with Highly Heterogeneous Responses In Vivo

Purpose: Preclinical tumor growth experiments often result in heterogeneous datasets that include growing, regressing, or stable growth profiles in the treatment and control groups. Such confounding intertumor variability may mask the true treatment effects especially when less aggressive treatment alternatives are being evaluated. Experimental design: We developed a statistical modeling approach in which the growing and poorly growing tumor categories were automatically detected by means of an expectation-maximization algorithm coupled within a mixed-effects modeling framework. The framework is implemented and distributed as an R package, which enables model estimation and statistical inference, as well as statistical power and precision analyses. Results: When applied to four tumor growth experiments, the modeling framework was shown to (i) improve the detection of subtle treatment effects in the presence of high within-group tumor variability; (ii) reveal hidden tumor subgroups associated with established or novel biomarkers, such as ERβ expression in a MCF-7 breast cancer model, which remained undetected with standard statistical analysis; (iii) provide guidance on the selection of sufficient sample sizes and most informative treatment periods; and (iv) offer flexibility to various cancer models, experimental designs, and treatment options. Model-based testing of treatment effect on the tumor growth rate (or slope) was shown as particularly informative in the preclinical assessment of treatment alternatives based on dietary interventions. Conclusions: In general, the modeling framework enables identification of such biologically significant differences in tumor growth profiles that would have gone undetected or had required considerably higher number of animals when using traditional statistical methods. Clin Cancer Res; 18(16); 4385–96. ©2012 AACR.