This paper deals with the Bayesian methods of comparing different types of dynamical structures for representing the given set of observations. Specifically, given that a given process y(\cdot) obeys one of r distinct stochastic or deterministic difference equations each involving a vector of unknown parameters, we compute the posterior probability that a set of observations {y(1),...,y(N)} obeys the i th equation, after making suitable assumptions about the prior probability distribution of the parameters in each equation. The difference equations can be nonlinear in the variable y but should be linear in the parameter vector in it. Once the posterior probability is known, we can find a decision rule to choose between the various structures so as to minimize the average value of a loss function. The optimum decision rule is asymptotically consistent and gives a quantitative explanation for the "principle of parsimony" often used in the construction of models from empirical data. The decision rule answers a wide variety of questions such as the advisability of a nonlinear transformation of data, the limitations of a model which yields a perfect fit to the data (i.e., zero residual variance), etc. The method can be used not only to compare different types of structures but also to determine a reliable estimate of spectral density of process. We compare the method in detail with the hypothesis testing method, and other methods and give a number of illustrative examples.
[1]
G. Yule.
On a Method of Investigating Periodicities in Disturbed Series, with Special Reference to Wolfer's Sunspot Numbers
,
1927
.
[2]
G. L. S. Shackle,et al.
PROBABILITY AND UNCERTAINTY
,
1949
.
[3]
Rangasami L. Kashyap,et al.
Prior probability and uncertainty
,
1971,
IEEE Trans. Inf. Theory.
[4]
J. Markel.
Digital inverse filtering-a new tool for formant trajectory estimation
,
1972
.
[5]
J. Burg.
THE RELATIONSHIP BETWEEN MAXIMUM ENTROPY SPECTRA AND MAXIMUM LIKELIHOOD SPECTRA
,
1972
.
[6]
D. E. Smylie,et al.
Analysis of Irregularities in the Earth's Rotation
,
1973
.
[7]
N. Andersen.
On the calculation of filter coefficients for maximum entropy spectral analysis
,
1974
.
[8]
H. Akaike.
A new look at the statistical model identification
,
1974
.
[9]
Rangasami L. Kashyap,et al.
Optimal feature selection and decision rules in classification problems with time series
,
1978,
IEEE Trans. Inf. Theory.