Adaptive Control in the Presence of Time-Varying Parameters

In the previous chapters two strands of literature have been brought together. The first strand is the research on time-varying parameters following a ‘Return to Normality’ model, viz. Harvey and Phillips (1982) and Swamy and Tinsley (1980).1 The second strand is the work on adaptive control in the tradition of Tse and Bar-Shalom (1973) and Kendrick (1981). As indicated in Chapter 2, given a time horizon of T periods, the goal was to find the values of control variables for period 0, period 1 and so on until period T−1 which minimized the objective functional, or cost-to-go, in an adaptive control framework. For each period in the time horizon, say period t, several trial, or search, values of the controls were tried.2 The optimal cost-to-go associated with each ‘search’ control was then calculated. Following Kendrick (1981, Ch. 10) the computations performed at time t were organized in three steps. First the nominal value for the parameters for period t through T was computed using their estimates for time t−1 and their ‘expected’ law of motion. The first ‘search’ control was also selected. Then the search for the optimal control was carried out. This step included: a) use the ‘search’ control and the nominal value for the parameters to get the projected states and covariances in period t+1; b) get the nominal path for the states and controls for the period t+1 through T by solving the certainty equivalence (CE) problem with the projected states as the initial condition; c) compute the Riccati matrices for periods t+1, ..., T; d) update and project the covariances from period t+1 given t, i.e. the covariance seen at point a) above, to T−1; e) calculate the approximate cost-to-go for the period t through T; f) choose a new ‘search’ control and repeat a)-f) until all `search’ points are evaluated; g) select the control which yields the minimum cost. After the selected control was applied to the system and the process moved one step forward in time, the estimates of the states and parameters were updated.