Developments of the total entropy utility function for the dual purpose of model discrimination and parameter estimation in Bayesian design

The total entropy utility function is considered for the dual purpose of Bayesian design for model discrimination and parameter estimation. A sequential design setting is proposed where it is shown how to efficiently estimate the total entropy utility for a wide variety of data types. Utility estimation relies on forming particle approximations to a number of intractable integrals which is afforded by the use of the sequential Monte Carlo algorithm for Bayesian inference. A number of motivating examples are considered for demonstrating the performance of total entropy in comparison to utilities for model discrimination and parameter estimation. The results suggest that the total entropy utility selects designs which are efficient under both experimental goals with little compromise in achieving either goal. As such, the total entropy utility is advocated as a general utility for Bayesian design in the presence of model uncertainty.

[1]  Anthony N. Pettitt,et al.  A Sequential Monte Carlo Algorithm to Incorporate Model Uncertainty in Bayesian Sequential Design , 2014 .

[2]  Shuguang Huang,et al.  OPTIMAL DESIGNS FOR GENERALIZED LINEAR MODELS WITH MULTIPLE DESIGN VARIABLES , 2011 .

[3]  Carl Lee,et al.  Constrained optimal designs for regressiom models , 1987 .

[4]  Anthony N. Pettitt,et al.  Fully Bayesian Experimental Design for Pharmacokinetic Studies , 2015, Entropy.

[5]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[6]  Mark A. Pitt,et al.  Adaptive Design Optimization: A Mutual Information-Based Approach to Model Discrimination in Cognitive Science , 2010, Neural Computation.

[7]  Holger Dette,et al.  Optimal discrimination designs , 2009, 0908.1912.

[8]  A. Atkinson,et al.  Optimum experimental design for discriminating between two rival models in the presence of prior information , 1991 .

[9]  Weichung Wang,et al.  A Modified Particle Swarm Optimization Technique for Finding Optimal Designs for Mixture Models , 2015, PloS one.

[10]  Carl Lee,et al.  Constrained optimal designs , 1988 .

[11]  S B Duffull,et al.  Optimal Design Criteria for Discrimination and Estimation in Nonlinear Models , 2009, Journal of biopharmaceutical statistics.

[12]  P. McCullagh,et al.  Generalized Linear Models , 1984 .

[13]  John Stufken,et al.  Locally ϕp-optimal designs for generalized linear models with a single-variable quadratic polynomial predictor , 2014 .

[14]  A. Atkinson,et al.  Optimal design : Experiments for discriminating between several models , 1975 .

[15]  D. Lindley.  On a Measure of the Information Provided by an Experiment , 1956 .

[16]  Peter Müller,et al.  Optimal sampling times in population pharmacokinetic studies , 2001 .

[17]  A Whitehead,et al.  Stopping rules for phase II studies. , 2001, British journal of clinical pharmacology.

[18]  W K Wong,et al.  Multiple-objective optimal designs. , 1998, Journal of biopharmaceutical statistics.

[19]  Xin-She Yang,et al.  Metaheuristic Optimization: Nature-Inspired Algorithms and Applications , 2013, Artificial Intelligence, Evolutionary Computing and Metaheuristics.

[20]  Holger Dette,et al.  Constrained D- and D1-optimal designs for polynomial regression , 2000 .

[21]  Johan Gabrielsson,et al.  Feedback modeling of non-esterified fatty acids in rats after nicotinic acid infusions , 2010, Journal of Pharmacokinetics and Pharmacodynamics.

[22]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[23]  Anthony C. Atkinson,et al.  DT-optimum designs for model discrimination and parameter estimation , 2008 .

[24]  William J. Hill,et al.  Discrimination Among Mechanistic Models , 1967 .

[25]  Sandro Ridella,et al.  Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithmCorrigenda for this article is available here , 1987, TOMS.

[26]  Christian P. Robert,et al.  Bayesian-Optimal Design via Interacting Particle Systems , 2006 .

[27]  P. Müller.  Simulation Based Optimal Design , 2005 .

[28]  James M. McGree,et al.  A pseudo-marginal sequential Monte Carlo algorithm for random effects models in Bayesian sequential design , 2016, Stat. Comput..

[29]  Mei-Mei Zen,et al.  Criterion-robust optimal designs for model discrimination and parameter estimation: Multivariate polynomial regression case , 2004 .

[30]  Holger Dette,et al.  Optimal Designs for Dose-Finding Studies , 2008 .

[31]  K. Mengersen,et al.  Adaptive Bayesian compound designs for dose finding studies , 2012 .

[32]  David C. Woods,et al.  Designs for Generalized Linear Models With Several Variables and Model Uncertainty , 2006, Technometrics.

[33]  Keying Ye,et al.  D-optimal designs for Poisson regression models , 2006 .

[34]  Holger Dette,et al.  Constrained D- and D1-optimal designs for polynomial regression , 2000 .

[35]  Szu Hui Ng,et al.  Design of follow‐up experiments for improving model discrimination and parameter estimation , 2004 .

[36]  W. J. Hill,et al.  A Joint Design Criterion for the Dual Problem of Model Discrimination and Parameter Estimation , 1968 .

[37]  N. Chopin.  A sequential particle filter method for static models , 2002 .

[38]  H. Rue,et al.  Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations , 2009 .

[39]  James M. McGree,et al.  Sequential Monte Carlo for Bayesian sequentially designed experiments for discrete data , 2013, Comput. Stat. Data Anal..

[40]  A. Atkinson,et al.  Designs for Generalized Linear Models , 2015, 1510.05253.

[41]  David M. Borth,et al.  A Total Entropy Criterion for the Dual Problem of Model Discrimination and Parameter Estimation , 1975 .

[42]  H. Dette,et al.  A Generalization of $D$- and $D_1$-Optimal Designs in Polynomial Regression , 1990 .

[43]  Anthony N. Pettitt,et al.  A Review of Modern Computational Algorithms for Bayesian Optimal Design , 2016 .

[44]  David C. Woods.  Designs for generalized linear models under model uncertainty , 2005 .

[45]  Say Beng Tan,et al.  Dose Finding Studies , 2009 .

[46]  Aleksandr Yakovlevich Khinchin,et al.  Mathematical foundations of information theory , 1959 .

[47]  Chiara Tommasi,et al.  Optimal designs for both model discrimination and parameter estimation , 2008 .

[48]  Christopher J Weir,et al.  Flexible Design and Efficient Implementation of Adaptive Dose-Finding Studies , 2007, Journal of biopharmaceutical statistics.

[49]  J. Eccleston,et al.  Compound Optimal Design Criteria for Nonlinear Models , 2008, Journal of biopharmaceutical statistics.

[50]  David C. Woods,et al.  Optimal designs for generalized non‐linear models with application to second‐harmonic generation experiments , 2009 .

[51]  Peter D. H. Hill,et al.  A Review of Experimental Design Procedures for Regression Model Discrimination , 1978 .

[52]  K. Chaloner,et al.  The Equivalence of Constrained and Weighted Designs in Multiple Objective Design Problems , 1996 .

[53]  Hovav A. Dror,et al.  Sequential Experimental Designs for Generalized Linear Models , 2008 .

[54]  G. Kitagawa.  Monte Carlo Filter and Smoother for Non-Gaussian Nonlinear State Space Models , 1996 .

[55]  A. M. Rosie,et al.  Information and Communication Theory , 2019 .

[56]  Anthony N Pettitt,et al.  Bayesian Experimental Design for Models with Intractable Likelihoods , 2013, Biometrics.

[57]  J. Lachin.  A review of methods for futility stopping based on conditional power , 2005, Statistics in medicine.

[58]  James M. McGree,et al.  Robust Designs for Poisson Regression Models , 2012, Technometrics.

[59]  Caterina May,et al.  MODEL SELECTION AND PARAMETER ESTIMATION IN NON-LINEAR NESTED MODELS: A SEQUENTIAL GENERALIZED DKL-OPTIMUM DESIGN , 2013 .

[60]  E. Läuter,et al.  Optimal multipurpose designs for regression models , 1976 .

[61]  Venkata V. Pavan Kumar,et al.  Evaluation of graphical diagnostics for assessing goodness of fit of logistic regression models , 2010, Journal of Pharmacokinetics and Pharmacodynamics.

[62]  A. Atkinson,et al.  The design of experiments for discriminating between two rival models , 1975 .

[63]  P. Moral,et al.  Sequential Monte Carlo samplers , 2002, cond-mat/0212648.