Log-density estimation in linear inverse problems

We estimate a probability density function p which is related by a linear operator K to a density function q in sequences of regular exponential families based on a random sample from q. In this paper deconvolution and positron emission tomography are considered. The logarithm of the density function is approximated by basis functions consisting of singular functions of K. While direct maximum likelihood (or minimum Kullback-Leibler) density estimation in exponential families selects the parameters to match the moments of the basis functions to the sample moments, in the inverse problem the moment of each singular function is related to a corresponding moment of the direct problem by a factor given by a singular value A, of K. Thus an appropriate analogue of the maximum likelihood estimate is obtained by matching moments with respect to p to 1/A, times the empirical moments associated with the sample from q. Bounds on the Kullback-Leibler distance between the true density and the estimators are obtained and rates of convergence are established for log-density functions having a measure of smoothness. The density estimator converges to the unknown density in the Kullback-Leibler sense and in the L2-sense at a rate determined not only by the order of smoothness of the log-density and the dimension of data but also by the decay rate of the singular values of the operator. A minimax lower bound for deconvolution is provided under certain conditions. Numerical examples using simulated data are provided to illustrate the finite-sample performance of the proposed method for deconvolution and positron emission tomography.

[1]  Wavelet density estimation by approximation of log-densities , 1996 .

[2]  J. D. Wilson,et al.  A smoothed EM approach to indirect estimation problems, with particular reference to stereology and emission tomography , 1990 .

[3]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[4]  B. Silverman,et al.  On the Estimation of a Probability Density Function by the Maximum Penalized Likelihood Method , 1982 .

[5]  Y. Vardi,et al.  From image deblurring to optimal investments : maximum likelihood solutions for positive linear inverse problems , 1993 .

[6]  John Rice,et al.  Deconvolution of Microfluorometric Histograms with B Splines , 1982 .

[7]  M. C. Jones,et al.  An orthogonal series density estimation approach to reconstructing positron emission tomography images , 1989 .

[8]  Vincent N. LaRiccia,et al.  Maximum Smoothed Likelihood Density Estimation for Inverse Problems , 1995 .

[9]  Bernard W. Silverman,et al.  Speed of Estimation in Positron Emission Tomography and Related Inverse Problems , 1990 .

[10]  R. Carroll,et al.  Deconvolving kernel density estimators , 1987 .

[11]  B. R. Crain,et al.  Estimation of Distributions Using Orthogonal Expansions , 1974 .

[12]  William H. Press,et al.  Numerical recipes in FORTRAN (2nd ed.): the art of scientific computing , 1992 .

[13]  E. Kolaczyk.  A Wavelet Shrinkage Approach to Tomographic Image Reconstruction , 1996 .

[14]  Sam Efromovich,et al.  Density Estimation for the Case of Supersmooth Measurement Error , 1997 .

[15]  F. O’Sullivan.  A Statistical Perspective on Ill-posed Inverse Problems , 1986 .

[16]  Ja-Yong Koo,et al.  B-spline deconvolution based on the EM algorithm , 1996 .

[17]  William Feller,et al.  An Introduction to Probability Theory and Its Applications , 1951 .

[18]  Jianqing Fan.  On the Optimal Rates of Convergence for Nonparametric Deconvolution Problems , 1991 .

[19]  C. J. Stone,et al.  Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[20]  C. J. Stone,et al.  Optimal Rates of Convergence for Nonparametric Estimators , 1980 .

[21]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[22]  Peter J. Bickel,et al.  Estimating linear functionals of a PET image , 1995, IEEE Trans. Medical Imaging.

[23]  D. Donoho,et al.  Geometrizing Rates of Convergence, III , 1991 .

[24]  P. Hall,et al.  Optimal Rates of Convergence for Deconvolving a Density , 1988 .

[25]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[26]  C. J. Stone.  Uniform Error Bounds Involving Logspline Models , 1989 .

[27]  B. R. Crain.  An Information Theoretic Approach to Approximating a Probability Distribution , 1977 .

[28]  S. Deans.  The Radon Transform and Some of Its Applications , 1983 .

[29]  C. J. Stone,et al.  Logspline Density Estimation for Censored Data , 1992 .

[30]  J. Neyman.  »Smooth test» for goodness of fit , 1937 .

[31]  Yuhong Yang,et al.  Information-theoretic determination of minimax rates of convergence , 1999 .

[32]  B. R. Crain.  Exponential Models, Maximum Likelihood Estimation, and the Haar Condition , 1976 .

[33]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[34]  D. Cox,et al.  Asymptotic techniques for use in statistics , 1989 .

[35]  Ja-Yong Koo,et al.  Optimal Rates of Convergence for Nonparametric Statistical Inverse Problems , 1993 .

[36]  A. Barron,et al.  APPROXIMATION OF DENSITY FUNCTIONS BY SEQUENCES OF EXPONENTIAL FAMILIES , 1991 .

[37]  B. R. Crain.  More on Estimation of Distributions Using Orthogonal Expansions , 1976 .

[38]  L. Shepp,et al.  A Statistical Model for Positron Emission Tomography , 1985 .

[39]  R. Z. Khasʹminskiĭ,et al.  Statistical estimation : asymptotic theory , 1981 .

[40]  M. Nussbaum.  Spline Smoothing in Regression Models and Asymptotic Efficiency in $L_2$ , 1985 .

[41]  C. J. Stone,et al.  A study of logspline density estimation , 1991 .

[42]  L. Mead,et al.  Maximum entropy in the problem of moments , 1984 .

[43]  D. Donoho.  Nonlinear Solution of Linear Inverse Problems by Wavelet–Vaguelette Decomposition , 1995 .

[44]  J. Koo.  Bivariate B-splines for tensor logspline density estimation , 1996 .

[45]  Bernard W. Silverman,et al.  Discretization effects in statistical inverse problems , 1991, J. Complex..

[46]  D. Donoho.  Statistical Estimation and Optimal Recovery , 1994 .

[47]  L. Devroye,et al.  Nonparametric Density Estimation: The L 1 View. , 1985 .

[48]  C. J. Stone,et al.  The Use of Polynomial Splines and Their Tensor Products in Multivariate Function Estimation , 1994 .

[49]  D. Nychka,et al.  CONVERGENCE RATES FOR REGULARIZED SOLUTIONS OF INTEGRAL EQUATIONS FROM DISCRETE NOISY DATA , 1989 .