Good, Bad and Optimal Experiments for Identification

Abstract: The topic of experiment design has attracted on-going interest for almost half a century. There is always a strong incentive to learn the most about a system with minimal perturbation to normal operations. In the early engineering literature there was considerable interest in nominal experiment design based on a-priori estimates of parameters. However, this can be precarious if the a-priori estimates are poor. This chapter will discuss more recent results on robust experiment design which, interalia, account for cases where the prior knowledge is less precise. Two illustrative case studies will be used to highlight the differences between “good, bad and optimal” experiments. In experiment design, as in many engineering problems, it is often better to have an approximate answer to the correct question than an “optimal” answer to the wrong question.

[1]  Michel Gevers,et al.  Identification For Control: Optimal Input Design With Respect To A Worst-Case $\nu$-gap Cost Function , 2002, SIAM J. Control. Optim..

[2]  Changbao Wu,et al.  Asymptotic inference from sequential design in a nonlinear situation , 1985 .

[3]  H. Chernoff.  Approaches in Sequential Design of Experiments , 1973 .

[4]  W. J. Studden,et al.  Optimal Experimental Designs , 1966 .

[5]  D. Titterington,et al.  Inference and sequential design , 1985 .

[6]  W. J. Studden,et al.  Theory Of Optimal Experiments , 1972 .

[7]  J. Kiefer.  General Equivalence Theory for Optimum Designs (Approximate Theory) , 1974 .

[8]  P. Whittle.  Some General Points in the Theory of Optimal Experimental Design , 1973 .

[9]  Robert Gardner,et al.  The Elements Of Integration , 1968 .

[10]  Håkan Hjalmarsson,et al.  From experiment design to closed-loop control , 2005, Autom..

[11]  Holger Dette,et al.  Standardized Maximin E-optimal Designs for the Michaelis Menten Model , 2002 .

[12]  Graham C. Goodwin,et al.  Robust optimal experiment design for system identification , 2007, Autom..

[13]  Leonard Eugene Dickson,et al.  Elementary Theory of Equations , 2008 .

[14]  J. Kiefer,et al.  The Equivalence of Two Extremum Problems , 1960, Canadian Journal of Mathematics.

[15]  Xavier Bombois,et al.  Least costly identification experiment for control , 2006, Autom..

[16]  H. Rosenbrock,et al.  Good, bad, or optimal? , 1971 .

[17]  G. Goodwin,et al.  Optimal test signal design for linear S.I.S.O. system identification , 1973 .

[18]  Viatcheslav B. Melas.  Optimal designs for exponential regression , 1978 .

[19]  Martin B. Zarrop,et al.  Optimal experiment design for dynamic system identification , 1977 .

[20]  D. B. Reid.  Optimal inputs for system identification , 1972 .

[21]  I. Glicksberg.  Minimax Theorem for Upper and Lower Semicontinuous Payoffs , 1950 .

[22]  S. Arimoto,et al.  Optimum input test signals for system identification—an information-theoretical approach , 1971 .

[23]  E. Walter,et al.  Robust experiment design via maximin optimization , 1988 .

[24]  David R. Cox.  Planning of Experiments , 1958 .

[25]  H. Wynn,et al.  Maximum entropy sampling and optimal Bayesian experimental design , 2000 .

[26]  V. Fedorov,et al.  Convex design theory 1 , 1980 .

[27]  R. Gagliardi.  Input selection for parameter identification in discrete systems , 1966, IEEE Transactions on Automatic Control.

[28]  Eric Walter,et al.  Identification of Parametric Models: from Experimental Data , 1997 .

[29]  Charles R. Johnson,et al.  Matrix analysis , 1985, Statistical Inference for Engineers and Data Scientists.

[30]  T. Başar,et al.  Dynamic Noncooperative Game Theory , 1982 .

[31]  Werner G. Müller,et al.  batch sequential design for a nonlinear estimation problem , 1989 .

[32]  Holger Dette,et al.  A note on maximin and Bayesian D-optimal designs in weighted polynomial regression , 2003 .

[33]  A. Wald.  On the Efficient Design of Statistical Investigations , 1943 .

[34]  Håkan Hjalmarsson,et al.  Robust Input Design Using Sum of Squares Constraints , 2006 .

[35]  W. Rudin.  Real and complex analysis , 1968 .

[36]  Michel Gevers,et al.  Minimizing the worst-case ν-gap by optimal input design , 2003 .

[37]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[38]  Michel Gevers.  Identification for Control: From the Early Achievements to the Revival of Experiment Design , 2005, CDC 2005.

[39]  K. Chaloner,et al.  Optimal Bayesian design applied to logistic regression experiments , 1989 .

[40]  Thomas R. Palfrey,et al.  Economical experiments: Bayesian efficient experimental design , 1996 .

[41]  K. Chaloner,et al.  Optimum experimental designs for properties of a compartmental model. , 1993, Biometrics.

[42]  V. Levadi.  Design of input signals for parameter estimation , 1966 .

[43]  H. Wynn.  Results in the Theory and Construction of D‐Optimum Experimental Designs , 1972 .

[44]  Graham C. Goodwin,et al.  UTILIZING PRIOR KNOWLEDGE IN ROBUST OPTIMAL EXPERIMENT DESIGN , 2006 .

[45]  S. Silvey,et al.  A sequentially constructed design for estimating a nonlinear parametric function , 1980 .