Dealing with observational data in control

Abstract There is growing interest in the use of control theory for interdisciplinary applications, where data may be sparse or missing, be non-uniformly sampled, have greater uncertainty, and where there is no opportunity to collect repeat measurements. In such applications, problems posed by observational data and the issue of missing or irregular data need to be considered. We present a review on dealing with observational, missing and irregular data for control applications. This considers the following issues: (i) how to identify a system model from observational data subject to missing measurements, (ii) how to determine control inputs when output data includes missing measurements, and (iii) how to ensure stability when future update times may be missed. Dealing with observational data and missing measurements is a key problem within the statistics literature, so we introduce statistical methods for dealing with this type of data. We aim to enable the integration of well-developed statistical methods of dealing with missing data into control theory. An example problem of using anticoagulants to control the blood clotting speed of patients is used throughout the paper.

[1]  H.-J. Tantau,et al.  Non-linear constrained MPC: Real-time implementation of greenhouse air temperature control , 2005 .

[2]  Eric B. Laber,et al.  A Robust Method for Estimating Optimal Treatment Regimes , 2012, Biometrics.

[3]  Jan Lunze,et al.  A state-feedback approach to event-based control , 2010, Autom..

[4]  Panos J. Antsaklis,et al.  On the model-based control of networked systems , 2003, Autom..

[5]  Onyebuchi A Arah,et al.  Bias Formulas for Sensitivity Analysis of Unmeasured Confounding for General Outcomes, Treatments, and Confounders , 2011, Epidemiology.

[6]  Yutaka Yamamoto,et al.  A retrospective view on sampled-data control systems , 1996 .

[7]  S. Murphy,et al.  Optimal dynamic treatment regimes , 2003 .

[8]  Manfred Morari,et al.  Linear offset-free Model Predictive Control , 2009, Autom..

[9]  Geert Molenberghs,et al.  Analyzing incomplete longitudinal clinical trial data. , 2004, Biostatistics.

[10]  Kenneth J. Hunt,et al.  Optimal control of heart rate during treadmill exercise , 2018 .

[11]  Geert Molenberghs,et al.  EVERY MISSING NOT AT RANDOM MODEL HAS GOT A MISSING AT RANDOM COUNTERPART WITH EQUAL FIT , 2008 .

[12]  Robin Henderson,et al.  Dynamic Analysis of Recurrent Event Data with Missing Observations, with Application to Infant Diarrhoea in Brazil , 2007 .

[13]  R. Yusupov,et al.  On the Application of Optimal Control Theory to Climate Engineering , 2017, 1709.05597.

[14]  J. Robins,et al.  Doubly Robust Estimation in Missing Data and Causal Inference Models , 2005, Biometrics.

[15]  M. Kenward,et al.  A comparison of multiple imputation and doubly robust estimation for analyses with missing data , 2006 .

[16]  Johan Nilsson,et al.  Real-Time Control Systems with Delays , 1998 .

[17]  R. Little,et al.  The prevention and treatment of missing data in clinical trials. , 2012, The New England journal of medicine.

[18]  D.-W. Gu,et al.  State estimation in the case of loss of observations , 2009, 2009 ICCAS-SICE.

[19]  Panos J. Antsaklis,et al.  MODEL-BASED NETWORKED CONTROL SYSTEMS – NECESSARY AND SUFFICIENT CONDITIONS FOR STABILITY , 2002 .

[20]  Emilia Fridman,et al.  Recent developments on the stability of systems with aperiodic sampling: An overview , 2017, Autom..

[21]  J. Pearl Causal inference in statistics: An overview , 2009 .

[22]  Michel Verhaegen,et al.  Application of a subspace model identification technique to identify LTI systems operating in closed-loop , 1993, Autom..

[23]  Mark J van der Laan,et al.  History-adjusted marginal structural models for estimating time-varying effect modification. , 2007, American journal of epidemiology.

[24]  D. D. Ruscio Model Predictive Control with Integral Action: A simple MPC algorithm , 2013 .

[25]  Carlos Bordons Alba,et al.  Model Predictive Control , 2012 .

[26]  D. Rubin Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[27]  Michael R Kosorok,et al.  Residual Weighted Learning for Estimating Individualized Treatment Rules , 2015, Journal of the American Statistical Association.

[28]  P. McCullagh Regression Models for Ordinal Data , 1980 .

[29]  Eugene M. Cliff,et al.  An optimal policy for a fish harvest , 1973 .

[30]  Petre Stoica,et al.  Decentralized Control , 2018, The Control Systems Handbook.

[31]  J. Hellendoorn,et al.  A macroscopic traffic flow model for integrated control of freeway and urban traffic networks , 2003, 42nd IEEE International Conference on Decision and Control (IEEE Cat. No.03CH37475).

[32]  Bo Hu,et al.  Stability analysis of digital feedback control systems with time-varying sampling periods , 2000, Autom..

[33]  N. Keiding,et al.  Estimation of dynamic treatment strategies for maintenance therapy of children with acute lymphoblastic leukaemia: an application of history‐adjusted marginal structural models , 2012, Statistics in medicine.

[34]  K. Poolla,et al.  Time varying optimal control with packet losses , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[35]  Marion Gilson,et al.  Instrumental variable methods for closed-loop system identification , 2005, Autom..

[36]  Daniel E. Rivera,et al.  An Improved Formulation of Hybrid Model Predictive Control With Application to Production-Inventory Systems , 2013, IEEE Transactions on Control Systems Technology.

[37]  Wpmh Maurice Heemels,et al.  Stability and stabilization of networked control systems , 2010 .

[38]  F. Ding,et al.  Least‐squares parameter estimation for systems with irregularly missing data , 2009 .

[39]  Leyla Gören Sümer,et al.  An Application of Robust Model Predictive Control with Integral Action , 2009 .

[40]  Sean R. Anderson,et al.  Biohybrid Control of General Linear Systems Using the Adaptive Filter Model of Cerebellum , 2015, Front. Neurorobot..

[41]  S. Nasraway,et al.  The Future is Now: Software-Guided Intensive Insulin Therapy in the Critically Ill , 2013, Journal of diabetes science and technology.

[42]  Evgeny Verbitskiy,et al.  Health technology assessment review: Computerized glucose regulation in the intensive care unit - how to create artificial control , 2009, Critical care.

[43]  Yan Shi,et al.  UNSCENTED KALMAN FILTERING FOR GREENHOUSE CLIMATE CONTROL SYSTEMS WITH MISSING MEASUREMENT , 2012 .

[44]  James Lam,et al.  A new delay system approach to network-based control , 2008, Autom..

[45]  Antonio Barreiro,et al.  Analysis of networked control systems with drops and variable delays , 2007, Autom..

[46]  Jean-Marie Aerts,et al.  Controlling horse heart rate as a basis for training improvement , 2008 .

[47]  Karl Johan Åström,et al.  Event Based Control , 2008 .

[48]  Karl Johan Åström,et al.  On limit cycles in event-based control systems , 2007, 2007 46th IEEE Conference on Decision and Control.

[49]  Douglas G. MacMartin,et al.  Dynamics of the coupled human–climate system resulting from closed-loop control of solar geoengineering , 2014, Climate Dynamics.

[50]  Manfred Morari,et al.  Nonlinear offset-free model predictive control , 2012, Autom..

[51]  João Pedro Hespanha,et al.  Exponential stability of impulsive systems with application to uncertain sampled-data systems , 2008, Syst. Control. Lett..

[52]  Marshall M Joffe,et al.  History-Adjusted Marginal Structural Models and Statically-Optimal Dynamic Treatment Regimens , 2005 .

[53]  B. Chakraborty,et al.  Statistical Methods for Dynamic Treatment Regimes: Reinforcement Learning, Causal Inference, and Personalized Medicine , 2013 .

[54]  C. James Taylor,et al.  Development of a grow-cell test facility for research into sustainable controlled-environment agriculture , 2016 .

[55]  R. Henderson,et al.  Optimal Dynamic Treatment Strategies with Protection Against Missed Decision Points , 2014 .

[56]  G. Thompson,et al.  Optimal Control Theory: Applications to Management Science and Economics , 2000 .

[57]  D. Rubin INFERENCE AND MISSING DATA , 1975 .

[58]  H G Watson,et al.  Guidelines on oral anticoagulation (warfarin): third edition – 2005 update , 2006, British journal of haematology.

[59]  Hisaya Fujioka Stability analysis of systems with aperiodic sample-and-hold devices , 2009, Autom..

[60]  Leonidas Dritsas,et al.  Robust stability analysis of Networked Systems with varying delays , 2009, 2009 European Control Conference (ECC).

[61]  Wpmh Maurice Heemels,et al.  Time-varying delays in control , 2006 .

[62]  Chris P. Underwood,et al.  HVAC Control Systems: Modelling, Analysis and Design , 1999 .

[63]  Stephanie T. Lanza,et al.  Control Engineering Methods for the Design of Robust Behavioral Treatments , 2017, IEEE Transactions on Control Systems Technology.

[64]  J. Aerts,et al.  Active control of the growth trajectory of broiler chickens based on online animal responses. , 2003, Poultry science.

[65]  Naresh N. Nandola,et al.  Optimized Treatment of Fibromyalgia Using System Identification and Hybrid Model Predictive Control. , 2014, Control engineering practice.

[66]  O. Aalen,et al.  Dynamic path analysis—a new approach to analyzing time-dependent covariates , 2006, Lifetime data analysis.

[67]  Guy A. Dumont,et al.  Robust control of depth of anesthesia , 2008 .

[68]  Lars Grüne,et al.  Using Nonlinear Model Predictive Control for Dynamic Decision Problems in Economics , 2015 .

[69]  Huazhen Fang,et al.  Kalman filter-based identification for systems with randomly missing measurements in a network environment , 2010, Int. J. Control.

[70]  Peter J. Gawthrop,et al.  Intermittent control: a computational theory of human control , 2011, Biological Cybernetics.

[71]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[72]  P. Young An instrumental variable method for real-time identification of a noisy process , 1970 .

[73]  Marko Bacic,et al.  Model predictive control , 2003 .

[74]  J. H. Westcott Control engineering and economic modelling: a collaboration aimed at improving control of the national economy , 1984 .

[75]  Karl Johan Åström,et al.  Numerical Identification of Linear Dynamic Systems from Normal Operating Records , 1965 .

[76]  Eleanor M Pullenayegum,et al.  Longitudinal data subject to irregular observation: A review of methods with a focus on visit processes, assumptions, and study design , 2016, Statistical methods in medical research.

[77]  Alberto Bemporad,et al.  Energy-aware robust model predictive control based on noisy wireless sensors , 2012, Autom..

[78]  Anastasios A. Tsiatis,et al.  Q- and A-learning Methods for Estimating Optimal Dynamic Treatment Regimes , 2012, Statistical science : a review journal of the Institute of Mathematical Statistics.

[79]  U. Shaked,et al.  Stability and guaranteed cost control of uncertain discrete delay systems , 2005 .

[80]  James M Robins,et al.  The International Journal of Biostatistics CAUSAL INFERENCE Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes , Part II : Proofs of Results , 2011 .

[81]  Yonina Rosen,et al.  Optimal ARMA parameter estimation based on the sample covariances for data with missing observations , 1989, IEEE Trans. Inf. Theory.

[82]  Donald B. Rubin,et al.  ‘Clarifying missing at random and related definitions, and implications when coupled with exchangeability’ , 2015 .

[83]  A. Dawid Causal Inference without Counterfactuals , 2000 .

[84]  Therese D. Pigott,et al.  A Review of Methods for Missing Data , 2001 .

[85]  J.P. Hespanha,et al.  Designing an observer-based controller for a network control system , 2005, Proceedings of the 44th IEEE Conference on Decision and Control.

[86]  S. Lewis,et al.  British Committee for Standards in Haematology , 1969 .

[87]  Peter J. Gawthrop,et al.  Intermittent model predictive control , 2007 .

[88]  P.J. Antsaklis,et al.  Model-Based Control with Intermittent Feedback , 2006, 2006 14th Mediterranean Conference on Control and Automation.

[89]  Joseph G. Ibrahim,et al.  Missing data methods in longitudinal studies: a review , 2009 .

[90]  Dragan Nesic,et al.  Input-output stability properties of networked control systems , 2004, IEEE Transactions on Automatic Control.

[91]  Vasileios Exadaktylos,et al.  Multi-objective performance optimisation for model predictive control by goal attainment , 2010, Int. J. Control.

[92]  M. Kenward,et al.  Every missingness not at random model has a missingness at random counterpart with equal fit , 2008 .

[93]  R. Henderson,et al.  Optimal dynamic treatment methods , 2011 .

[94]  W. P. M. H. Heemels,et al.  Analysis of event-driven controllers for linear systems , 2008, Int. J. Control.

[95]  W. P. M. H. Heemels,et al.  Controller synthesis for networked control systems , 2010, Autom..

[96]  Antonio Sala,et al.  Computer control under time-varying sampling period: An LMI gridding approach , 2005, Autom..

[97]  P. Holland Statistics and Causal Inference , 1985 .

[98]  Ivan Markovsky Exact system identification with missing data , 2013, 52nd IEEE Conference on Decision and Control.

[99]  P. Diggle Analysis of Longitudinal Data , 1995 .

[100]  Fan Li,et al.  Causal Inference: A Missing Data Perspective , 2017, 1712.06170.

[101]  Eyal Dassau,et al.  Event-Triggered Model Predictive Control for Embedded Artificial Pancreas Systems , 2018, IEEE Transactions on Biomedical Engineering.

[102]  V. Didelez,et al.  Ignorability for general longitudinal data , 2017, Biometrika.

[103]  R. Hovorka,et al.  Nonlinear model predictive control of glucose concentration in subjects with type 1 diabetes. , 2004, Physiological measurement.

[104]  David A Stephens,et al.  Simulating sequential multiple assignment randomized trials to generate optimal personalized warfarin dosing strategies , 2014, Clinical trials.

[105]  Jessica K. Barrett,et al.  Doubly Robust Estimation of Optimal Dynamic Treatment Regimes , 2014, Statistics in biosciences.

[106]  Guy Albert Dumont,et al.  Introduction to Automated Drug Delivery in Clinical Anesthesia , 2005, Eur. J. Control.

[107]  Masayoshi Tomizuka,et al.  Multirate estimation and control under time-varying data sampling with applications to information storage devices , 1995, Proceedings of 1995 American Control Conference - ACC'95.

[108]  Peter C. Young,et al.  True Digital Control: Statistical Modelling and Non-Minimal State Space Design , 2013 .

[109]  Roderick J. A. Little,et al.  Conditions for Ignoring the Missing-Data Mechanism in Likelihood Inferences for Parameter Subsets , 2017 .

[110]  Dragan Nesic,et al.  Quadratic stabilization of linear networked control systems via simultaneous protocol and controller design , 2007, Autom..

[111]  Troy Day,et al.  Optimal control of epidemics with limited resources , 2011, Journal of mathematical biology.

[112]  Alf Isaksson,et al.  Identification of ARX-models subject to missing data , 1993, IEEE Trans. Autom. Control..

[113]  James M. Robins,et al.  Optimal Structural Nested Models for Optimal Sequential Decisions , 2004 .

[114]  Guy A. Dumont,et al.  Closed-Loop Control of Anesthesia - a Review , 2012 .

[115]  Geert Verbeke,et al.  Handbooks of Modern Statistical Methods Longitudinal Data Analysis , 2008 .

[116]  Peter C. Young,et al.  Recursive Estimation and Time-Series Analysis: An Introduction , 1984 .

[117]  D. Melzer,et al.  Adjusting for unmeasured confounding in nonrandomized longitudinal studies: a methodological review , 2017, Journal of clinical epidemiology.

[118]  R Henderson,et al.  Joint modelling of longitudinal measurements and event time data. , 2000, Biostatistics.

[119]  Peter J. Gawthrop,et al.  Event-driven intermittent control , 2009, Int. J. Control.

[120]  Mo-Yuen Chow,et al.  Networked Control System: Overview and Research Trends , 2010, IEEE Transactions on Industrial Electronics.

[121]  Sina Mirsaidi,et al.  LMS-like AR modeling in the case of missing observations , 1997, IEEE Trans. Signal Process..

[122]  Rik Pintelon,et al.  Frequency domain system identification with missing data , 1998, Proceedings of the 37th IEEE Conference on Decision and Control (Cat. No.98CH36171).

[123]  Leonid Mirkin On the use of time-varying delay to represent sample-and-hold circuits , 2007, 2007 46th IEEE Conference on Decision and Control.

[124]  Geert Molenberghs,et al.  Missing Data in Clinical Studies , 2007 .

[125]  H. Behncke Optimal control of deterministic epidemics , 2000 .

[126]  Hisaya Fujioka,et al.  A Discrete-Time Approach to Stability Analysis of Systems With Aperiodic Sample-and-Hold Devices , 2009, IEEE Transactions on Automatic Control.

[127]  Peter C. Young,et al.  Stabilizing global mean surface temperature: A feedback control perspective , 2009, Environ. Model. Softw..

[128]  Manfred Morari,et al.  Robust constrained model predictive control using linear matrix inequalities , 1994, Proceedings of 1994 American Control Conference - ACC '94.

[129]  José Antonio López Orozco,et al.  An Asynchronous, Robust, and Distributed Multisensor Fusion System for Mobile Robots , 2000, Int. J. Robotics Res..

[130]  Wei Zhang,et al.  Stability of networked control systems , 2001 .

[131]  D. Rubin Causal Inference Using Potential Outcomes , 2005 .

[132]  Daniel E. Quevedo,et al.  Self-Triggered Model Predictive Control for Network Scheduling and Control , 2012 .

[133]  Guangming Xie,et al.  Stabilization of networked control systems with time-varying network-induced delay , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[134]  Paulo Tabuada,et al.  An introduction to event-triggered and self-triggered control , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[135]  Lennart Ljung,et al.  An iterative method for identification of ARX models from incomplete data , 2000, Proceedings of the 39th IEEE Conference on Decision and Control (Cat. No.00CH37187).

[136]  Huaicheng Yan,et al.  An overview of networked control of complex dynamic systems , 2014 .

[137]  Phil Ansell,et al.  Regret‐Regression for Optimal Dynamic Treatment Regimes , 2010, Biometrics.

[138]  Panos J. Antsaklis,et al.  Stability of model-based networked control systems with time-varying transmission times , 2004, IEEE Transactions on Automatic Control.

[139]  Henrik Gollee,et al.  Identification of intermittent control in man and machine , 2012, Journal of The Royal Society Interface.

[140]  Roman Hovorka,et al.  Multicentric, randomized, controlled trial to evaluate blood glucose control by the model predictive control algorithm versus routine glucose management protocols in intensive care unit patients. , 2006, Diabetes care.

[141]  Paul M.J. Van den Hof,et al.  Closed-Loop Issues in System Identification , 1997 .

[142]  George J. Pappas,et al.  Analysis and Control of Epidemics: A Survey of Spreading Processes on Complex Networks , 2015, IEEE Control Systems.

[143]  Torsten Söderström,et al.  Identification of continuous-time AR processes from unevenly sampled data , 2002, Autom..

[144]  M. Hoagland,et al.  Feedback Systems An Introduction for Scientists and Engineers SECOND EDITION , 2015 .

[145]  Dan Jackson,et al.  What Is Meant by "Missing at Random"? , 2013, 1306.2812.

[146]  Y. Tipsuwan,et al.  Control methodologies in networked control systems , 2003 .

[147]  K. Åström,et al.  Comparison of Riemann and Lebesgue sampling for first order stochastic systems , 2002, Proceedings of the 41st IEEE Conference on Decision and Control, 2002..

[148]  Philip E. Pare,et al.  Feedback linearization control methods for accurate leaf photosynthesis measurements , 2017, 2017 American Control Conference (ACC).

[149]  M. Morari,et al.  Multitasked closed-loop control in anesthesia , 2001, IEEE Engineering in Medicine and Biology Magazine.

[150]  Donglin Zeng,et al.  Estimating Individualized Treatment Rules Using Outcome Weighted Learning , 2012, Journal of the American Statistical Association.

[151]  Richard H. Jones,et al.  Maximum Likelihood Fitting of ARMA Models to Time Series With Missing Observations , 1980 .

[152]  Zidong Wang,et al.  A survey of event-based strategies on control and estimation , 2014 .

[153]  J.P. Hespanha,et al.  On the robust stability and stabilization of sampled-data systems: A hybrid system approach , 2006, Proceedings of the 45th IEEE Conference on Decision and Control.

[154]  Roberto Sanchis,et al.  Scarce Data Operating Conditions: Process Model Identification , 1997 .

[155]  Roberto Sanchis,et al.  Recursive identification under scarce measurements - convergence analysis , 2002, Autom..

[156]  Karl-Erik Årzén,et al.  A simple event-based PID controller , 1999 .

[157]  João Pedro Hespanha,et al.  A Survey of Recent Results in Networked Control Systems , 2007, Proceedings of the IEEE.

[158]  M. J. van der Laan,et al.  Super-Learning of an Optimal Dynamic Treatment Rule , 2016, The international journal of biostatistics.

[159]  K. Åström,et al.  Comparison of Periodic and Event Based Sampling for First-Order Stochastic Systems , 1999 .

[160]  S. Lipsitz,et al.  Missing-Data Methods for Generalized Linear Models , 2005 .

[161]  D. Rivera,et al.  Using engineering control principles to inform the design of adaptive interventions: a conceptual introduction. , 2007, Drug and alcohol dependence.

[162]  B. Bobrovsky,et al.  Computer-controlled heart rate increase by isoproterenol infusion: mathematical modeling of the system. , 1999, The American journal of physiology.

[163]  Johan Nilsson,et al.  Stochastic Analysis and Control of Real-Time Systems with Random Time Delays , 1996 .

[164]  Dimos V. Dimarogonas,et al.  Novel event-triggered strategies for Model Predictive Controllers , 2011, IEEE Conference on Decision and Control and European Control Conference.

[165]  T. Speed,et al.  On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9 , 1990 .

[166]  Mara Guinaldo Losada,et al.  Asynchronous Control for Networked Systems , 2015 .

[167]  J. Robins,et al.  The International Journal of Biostatistics CAUSAL INFERENCE Dynamic Regime Marginal Structural Mean Models for Estimation of Optimal Dynamic Treatment Regimes , Part I : Main Content , 2011 .

[168]  John W. Nicklow Discrete-Time Optimal Control for Water Resources Engineering and Management , 2000 .

[169]  D. Rubin [On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9.] Comment: Neyman (1923) and Causal Inference in Experiments and Observational Studies , 1990 .

[170]  Jean-Marie Aerts,et al.  Control of Nonlinear Biological Systems by Non-minimal State Variable Feedback , 2014 .

[171]  Tai C Yang,et al.  Networked control system: a brief survey , 2006 .