Stochastic estimation of a mixture of normal density functions using an information criterion

A stochastic approximation algorithm is developed for estimating a mixture of normal density functions with unknown means and unknown variances. The algorithm minimizes an information criterion that has interesting properties for density approximations. The conditions on the convergence of this nonlinear estimation algorithm are discussed, and a numerical example is presented.

[1]  R. A. Leibler,et al.  On Information and Sufficiency , 1951 .

[2]  J. Kiefer,et al.  Stochastic Estimation of the Maximum of a Regression Function , 1952 .

[3]  J. Blum.  Multidimensional Stochastic Approximation Methods , 1954 .

[4]  Solomon Kullback,et al.  Information Theory and Statistics , 1960 .

[5]  A. Prékopa,et al.  Decomposition of superpositions of distribution functions , 1962 .

[6]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[7]  H. Teicher.  Identifiability of Finite Mixtures , 1963 .

[8]  David B. Cooper,et al.  Nonsupervised Adaptive Signal Detection and Pattern Recognition , 1964, Inf. Control..

[9]  D. Sakrison.  Efficient recursive estimation; application to estimating the parameters of a covariance function , 1965 .

[10]  Y. T. CHIEN,et al.  On Bayesian Learning and Stochastic Approximation , 1967, IEEE Trans. Syst. Sci. Cybern..

[11]  D. Lainiotis,et al.  Minimum mean square error approximation of unknown probability distribution functions , 1968 .

[12]  John C. Hancock,et al.  An optimum decision-directed scheme for Gaussian mixtures , 1968, IEEE Trans. Inf. Theory.

[13]  S. Yakowitz,et al.  On the Identifiability of Finite Mixtures , 1968 .

[14]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[15]  Rangasami L. Kashyap,et al.  Estimation of probability density and distribution functions , 1968, IEEE Trans. Inf. Theory.

[16]  T. Young,et al.  On estimation of a mixture of normal density functions , 1969 .

[17]  T. Young,et al.  Stochastic Signal Representation , 1969 .

[18]  Solomon Kullback,et al.  Information Theory and Statistics , 1970, The Mathematical Gazette.