Distribution-Free Prediction Sets

This article introduces a new approach to prediction by bringing together two different nonparametric ideas: distribution-free inference and nonparametric smoothing. Specifically, we consider the problem of constructing nonparametric tolerance/prediction sets. We start from the general conformal prediction approach, and we use a kernel density estimator as a measure of agreement between a sample point and the underlying distribution. The resulting prediction set is shown to be closely related to plug-in density level sets with carefully chosen cutoff values. Under standard smoothness conditions, we get an asymptotic efficiency result that is near optimal for a wide range of function classes. But the coverage is guaranteed whether or not the smoothness conditions hold and regardless of the sample size. The performance of our method is investigated through simulation studies and illustrated in a real data example.

[1]  S. S. Wilks.  Determination of Sample Sizes for Setting Tolerance Limits , 1941 .

[2]  Abraham Wald,et al.  An Extension of Wilks' Method for Setting Tolerance Limits , 1943 .

[3]  J. Tukey.  Non-Parametric Estimation II. Statistically Equivalent Blocks and Tolerance Regions--The Continuous Case , 1947 .

[4]  I. Guttman,et al.  Statistical Tolerance Regions: Classical and Bayesian , 1970 .

[5]  John Aitchison,et al.  Statistical Prediction Analysis , 1975 .

[6]  Shoutir Kishore Chatterjee,et al.  Asymptotically Minimal Multivariate Tolerance Sets , 1980 .

[7]  Prakasa Rao.  Nonparametric functional estimation , 1983 .

[8]  W. Polonik.  Measuring Mass Concentrations and Estimating Density Contour Clusters-An Excess Mass Approach , 1995 .

[9]  Rob J Hyndman,et al.  Computing and Graphing Highest Density Regions , 1996 .

[10]  Jon A. Wellner,et al.  Weak Convergence and Empirical Processes: With Applications to Statistics , 1996 .

[11]  Guenther Walther,et al.  Monte Carlo sampling in dual space for approximating the empirical halfspace distance , 1997 .

[12]  A. Tsybakov.  On nonparametric estimation of density level sets , 1997 .

[13]  W. Polonik.  Minimum volume sets and generalized quantile processes , 1997 .

[14]  C. Loader.  Bandwidth selection: classical or plug-in? , 1999 .

[15]  A. Di Bucchianico,et al.  Smallest nonparametric tolerance regions , 2001 .

[16]  J. A. Cuesta-Albertos,et al.  Convergence rates in nonparametric estimation of level sets , 2001 .

[17]  E. Giné,et al.  Rates of strong uniform consistency for multivariate kernel density estimators , 2002 .

[18]  A. Baíllo.  Total Error in a Plug-in Estimator of Level Sets , 2003 .

[19]  Don R. Hush,et al.  A Classification Framework for Anomaly Detection , 2005, J. Mach. Learn. Res..

[20]  Robert D. Nowak,et al.  Learning Minimum Volume Sets , 2005, J. Mach. Learn. Res..

[21]  B. Cadre.  Kernel estimation of density level sets , 2005, math/0501221.

[22]  W. Gasarch,et al.  The Book Review Column 1 Coverage Untyped Systems Simple Types Recursive Types Higher-order Systems General Impression 3 Organization, and Contents of the Book , 2022 .

[23]  Robert D. Nowak,et al.  Minimax Optimal Level-Set Estimation , 2007, IEEE Transactions on Image Processing.

[24]  A. Tsybakov,et al.  Fast learning rates for plug-in classifiers , 2007, 0708.2321.

[25]  Jun Li,et al.  Multivariate spacings based on data depth: I. Construction of nonparametric multivariate tolerance regions , 2008, 0806.2970.

[26]  Ying Wei.  An Approach to Multivariate Covariate-Dependent Quantile Contours With Application to Bivariate Conditional Growth Charts , 2008 .

[27]  Vladimir Vovk,et al.  A tutorial on conformal prediction , 2007, J. Mach. Learn. Res..

[28]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[29]  Bruno Pelletier,et al.  Clustering by estimation of density level sets at a fixed probability , 2009 .

[30]  P. Rigollet,et al.  Optimal rates for plug-in estimators of density level sets , 2006, math/0611473.

[31]  Venkatesh Saligrama,et al.  Anomaly Detection with Score functions based on Nearest Neighbor Graphs , 2009, NIPS.

[32]  A. Gammerman,et al.  On-line predictive linear regression , 2005, math/0511522.

[33]  A. Rinaldo,et al.  Generalized density clustering , 2009, 0907.3454.

[34]  Yu Ding,et al.  A Computable Plug-In Estimator of Minimum Volume Sets for Novelty Detection , 2010, Oper. Res..

[35]  M. P. Wand,et al.  Asymptotics and optimal bandwidth selection for highest density region estimation , 2010, 1010.0591.

[36]  Alfred O. Hero,et al.  Efficient anomaly detection using bipartite k-NN graphs , 2011, NIPS.

[37]  Jens Perch Nielsen,et al.  Do-Validation for Kernel Density Estimation , 2011 .

[38]  Vincent Kanade,et al.  Clustering Algorithms , 2021, Wireless RF Energy Transfer in the Massive IoT Era.