Maximum Likelihood Method

A classical problem in the statistical decision theory is to estimate the probability distribution of a random vector X given its independent observations \(x_{1},\ldots,x_{n}\). Often it is assumed that the probability distribution comes from some family of functions parametrized by a set of parameters \(\theta _{1},\ldots,\theta _{m}\), so that in this case, the problem is reduced to estimating \(\theta _{1},\ldots,\theta _{m}\) and is called parametric estimation. However, if no specific family of distributions is assumed, i.e., the probability distribution can not be completely defined by a finite number of parameters, the problem is called nonparametric estimation.