Orthogonal series density estimation

Orthogonal series density estimation is a powerful nonparametric estimation methodology that allows one to analyze and present data at hand without any prior opinion about shape of an underlying density. The idea of construction of an adaptive orthogonal series density estimator is explained on the classical example of a direct sample from a univariate density. Data‐driven estimators, which have been used for years, as well as recently proposed procedures, are reviewed. Orthogonal series estimation is also known for its sharp minimax properties which are explained. Furthermore, applications of the orthogonal series methodology to more complicated settings, including censored and biased data as well as estimation of the density of regression errors and the conditional density, are also presented. Copyright © 2010 John Wiley & Sons, Inc.

[1]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[2]  S. Schwartz.  Estimation of Probability Density by an Orthogonal Series , 1967 .

[3]  R. Kronmal,et al.  The Estimation of Probability Densities and Cumulatives by Fourier Series Methods , 1968 .

[4]  Geoffrey S. Watson,et al.  Density Estimation by Orthogonal Series , 1969 .

[5]  E. Wegman.  Nonparametric probability density estimation , 1972 .

[6]  E. Wegman.  Nonparametric Probability Density Estimation: I. A Summary of Available Methods , 1972 .

[7]  B. R. Crain.  A Note on Density Estimation Using Orthogonal Expansions , 1973 .

[8]  B. R. Crain,et al.  Estimation of Distributions Using Orthogonal Expansions , 1974 .

[9]  G. Wahba.  Optimal Convergence Properties of Variable Knot, Kernel, and Orthogonal Series Methods for Density Estimation. , 1975 .

[10]  R. Kronmal,et al.  An Introduction to the Implementation and Theory of Nonparametric Density Estimation , 1976 .

[11]  G. Walter.  Properties of Hermite Series Estimation of Probability Density , 1977 .

[12]  Rui J. P. de Figueiredo,et al.  An Adaptive Orthogonal-Series Estimator for Probability Density Functions , 1978 .

[13]  Peter Hall.  Estimating a density on the positive half line by the method of orthogonal series , 1980 .

[14]  I. Good,et al.  Density Estimation and Bump-Hunting by the Penalized Likelihood Method Exemplified by Scattering and Meteorite Data , 1980 .

[15]  G. Wahba.  Data-Based Optimal Smoothing of Orthogonal Series Density Estimates , 1981 .

[16]  R. Z. Khasʹminskiĭ,et al.  Statistical estimation : asymptotic theory , 1981 .

[17]  P. Hall.  On Trigonometric Series Estimates of Densities , 1981 .

[18]  P. Hall.  Orthogonal Series Methods for Both Qualitative and Quantitative Data , 1983 .

[19]  J. Hart.  On the choice of a truncation point in fourier series density estimation , 1985 .

[20]  Peter J. Diggle,et al.  The Selection of Terms in an Orthogonal Series Density Estimator , 1986 .

[21]  Lesław Gajek,et al.  On Improving Density Estimators which are not Bona Fide Functions , 1986 .

[22]  S. Efroimovich.  Nonparametric Estimation of a Density of Unknown Smoothness , 1986 .

[23]  D. W. Scott,et al.  Biased and Unbiased Cross-Validation in Density Estimation , 1987 .

[24]  P. Hall.  Cross—validation and the smoothing of orthogonal series density estimators , 1987 .

[25]  L. Devroye,et al.  Nonparametric density estimation : the L[1] view , 1987 .

[26]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[27]  Harrie Hendriks,et al.  Nonparametric Estimation of a Probability Density on a Riemannian Manifold Using Fourier Expansions , 1990 .

[28]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[29]  Stephen T. Buckland,et al.  Fitting Density Functions with Polynomials , 1992 .

[30]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[31]  M. Wand,et al.  EXACT MEAN INTEGRATED SQUARED ERROR , 1992 .

[32]  M. C. Jones,et al.  Model-Free Curve Estimation , 1993 .

[33]  G. Walter.  Wavelets and other orthogonal systems with applications , 1994 .

[34]  Robert M. Wharton.  Model-Free Curve Estimation , 1995 .

[35]  S. Efromovich.  Adaptive orthogonal series density estimation for small samples , 1996 .

[36]  I. Johnstone,et al.  Density estimation by wavelet thresholding , 1996 .

[37]  Sam Efromovich,et al.  Density Estimation for the Case of Supersmooth Measurement Error , 1997 .

[38]  P. Hall,et al.  Block threshold rules for curve estimation using kernel and wavelet methods , 1998 .

[39]  William Alexander,et al.  Nonparametric Smoothing and Lack-of-Fit Tests , 1999, Technometrics.

[40]  Yuhong Yang.  Mixing Strategies for Density Estimation , 2000 .

[41]  N. Čencov.  Statistical Decision Rules and Optimal Inference , 2000 .

[42]  Lianfen Qian,et al.  Nonparametric Curve Estimation: Methods, Theory, and Applications , 1999, Technometrics.

[43]  S. Efromovich.  Density Estimation Under Random Censorship and Order Restrictions , 2001 .

[44]  Luc Devroye,et al.  Combinatorial methods in density estimation , 2001, Springer series in statistics.

[45]  D. Zerom Godefay,et al.  On conditional density estimation , 2003 .

[46]  Olivier Catoni,et al.  Statistical learning theory and stochastic optimization , 2004 .

[47]  Sam Efromovich.  Density estimation for biased data , 2004 .

[48]  Alexandre B. Tsybakov,et al.  Aggregation of density estimators and dimension reduction , 2005 .

[49]  T. Cai,et al.  Block thresholding for density estimation: local and global adaptivity , 2005 .

[50]  S. Efromovich.  Estimation of the density of regression errors , 2005, math/0602242.

[51]  PHIL IPPE R IGOLLET,et al.  Adaptive density estimation using the blockwise Stein method , 2006 .

[52]  A. Tsybakov,et al.  Linear and convex aggregation of density estimators , 2006, math/0605292.

[53]  Sam Efromovich.  Optimal nonparametric estimation of the density of regression errors with finite support , 2007 .

[54]  Sam Efromovich.  Adaptive estimation of error density in nonparametric regression with small sample size , 2007 .

[55]  P. Massart,et al.  Concentration inequalities and model selection , 2007 .

[56]  Sam Efromovich,et al.  CONDITIONAL DENSITY ESTIMATION IN A REGRESSION SETTING , 2007, 0803.2984.

[57]  S. Efromovich.  Adaptive estimation of and oracle inequalities for probability densities and characteristic functions , 2008, 0806.2909.

[58]  S. Efromovich.  Lower bound for estimation of Sobolev densities of order less 12 , 2009 .

[59]  Richard A. Davis,et al.  Remarks on Some Nonparametric Estimates of a Density Function , 2011 .