Model evaluation in nonlinear mixed effect models

• Several simulation-based metrics developed over the last decade: – Visual Predictive Checks (VPC) [1] – prediction discrepancies (pd) [2] – normalised prediction distribution errors (npde) [3] • Assumptions – model MB has been built using a building dataset B – null hypothesis: this model can be used to describe the data collected in a validation dataset V (=B in internal evaluation) • General class of Posterior Predictive Check (PPC), born in the Bayesian world – model MB used to simulate data according to the design of V – compare a statistic computed on the real data in V to the distribution of the statistic obtained through the simulations – here plug-in approach (ignoring uncertainty)