A classification of algorithms for ARMA models and ladder realizations

Applications of linear systems modeling have recently developed quite rapidly in speech modeling, seismic data processing, and other areas. Due to the diversity of these developments, there exists a plethora of methods for estimating the parameters of linear models given input-ouput data, transfer functions, or covariance functions. This paper attempts a systematic classification of existing least-squares modeling methods. Within this framework, we shall point out some recently developed algorithms that have many computational advantages over existing ones. In particular, the methods of interest will be classified according to how the input/output data is acessed and according to its type. Data can be accessed either sequentially or in blocks; the data can be either input/output signals, transfer functions, or covariance functions. Since we consider state-space, autoregressive-moving average models, and the related ladder realizations, we shall distinguish the following three classes of algorithms: Riccali or square-root type methods, recently developed "fast" algorithms, and their ladder forms. While the first class typically requires computations of O(n3) or O(n2) with n equal to the number of model parameters, the "last" forms only require operations and storage of O(n). The ladder realizations have several advantages, such as lowest complexity and their stability "by inspection" properties. In the appendices, we present an example of our new exact least-squares recursions for ladder forms, and show how to obtain stable partial minimal realizations of the joint impulse response - and covariance - matching type.