Maximum Likelihood Estimation for Lossy Data Compression

In lossless data compression, given a sequence of observations (Xn)n≥1 and a family of probability distributions {Qθ}θ∈Θ, the estimators (θn)n≥1 obtained by minimizing the ideal Shannon code-lengths over the family {Qθ}θ∈Θ, θn := arg min θ∈Θ [ − logQθ(X 1 ) ] , whereXn 1 := (X1, X2, . . . , Xn), coincide with the classical maximum-likelihood estimators (MLEs). In the corresponding lossy compression setting, the ideal Shannon code-lengths are approximately − logQθ(B(X 1 , D)) bits, where B(Xn 1 , D) is the distortion-ball of radius D around the source sequence Xn 1 . In this work we consider the analogous estimators obtained by minimizing these lossy code-lengths, θn := arg min θ∈Θ [ − logQθ(B(X 1 , D)) ] . The θn are a lossy version of the MLEs, which we call “lossy MLEs”. We investigate the strong consistency of lossy MLEs when the Qθ are i.i.d. and the sequence (Xn)n≥1 is stationary and ergodic.