Prediction error quantification through probabilistic scaling -- EXTENDED VERSION

In this paper, we address the probabilistic error quantification of a general class of prediction methods. We consider a given prediction model and show how to obtain, through a sample-based approach, a probabilistic upper bound on the absolute value of the prediction error. The proposed scheme is based on a probabilistic scaling methodology in which the number of required randomized samples is independent of the complexity of the prediction model. The methodology is extended to address the case in which the probabilistic uncertain quantification is required to be valid for every member of a finite family of predictors. We illustrate the results of the paper by means of a numerical example.

[1]  Mark Huber,et al.  Halving the Bounds for the Markov, Chebyshev, and Chernoff Inequalities Using Smoothing , 2018, Am. Math. Mon..

[2]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[3]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[4]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[5]  Giuseppe Carlo Calafiore,et al.  Random Convex Programs , 2010, SIAM J. Optim..

[6]  Naomi S. Altman,et al.  Quantile regression , 2019, Nature Methods.

[7]  Vladimir Vovk,et al.  A tutorial on conformal prediction , 2007, J. Mach. Learn. Res..

[8]  Vladimir Vovk,et al.  Conformal Prediction for Reliable Machine Learning: Theory, Adaptations and Applications , 2014 .

[9]  Fabrizio Dabbene,et al.  Chance constrained sets approximation: A probabilistic scaling approach - EXTENDED VERSION , 2021, Autom..

[10]  Marco C. Campi,et al.  The exact feasibility of randomized solutions of robust convex programs , 2008 .

[11]  Julio R. Banga,et al.  A Comparison of Methods for Quantifying Prediction Uncertainty in Systems Biology , 2019, IFAC-PapersOnLine.

[12]  Eduardo F. Camacho,et al.  Robust Design Through Probabilistic Maximization , 2018 .

[13]  Bruno Pelletier.  Kernel density estimation on Riemannian manifolds , 2005 .

[14]  Marco C. Campi,et al.  On a class of interval predictor models with universal reliability , 2019, Autom..

[15]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[16]  Roberto Tempo,et al.  Randomized methods for design of uncertain systems: Sample complexity and sequential algorithms , 2013, Autom..

[17]  R. Tempo,et al.  Probabilistic robustness analysis: explicit bounds for the minimum number of samples , 1996, Proceedings of 35th IEEE Conference on Decision and Control.

[18]  Vladimir Vovk,et al.  Kernel Ridge Regression , 2013, Empirical Inference.

[19]  Michael Rabadi,et al.  Kernel Methods for Machine Learning , 2015 .

[20]  Mohammad Ahsanullah,et al.  An Introduction to Order Statistics , 2013 .

[21]  Fabrizio Dabbene,et al.  Computationally efficient stochastic MPC: a probabilistic scaling approach , 2020, 2020 IEEE Conference on Control Technology and Applications (CCTA).

[22]  Giuseppe Carlo Calafiore,et al.  Interval predictor models: Identification and reliability , 2009, Autom..

[23]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[24]  Teodoro Alamo,et al.  Probabilistic performance validation of deep learning‐based robust NMPC controllers , 2019, International Journal of Robust and Nonlinear Control.

[25]  Fabrizio Dabbene,et al.  A probabilistic validation approach for penalty function design in Stochastic Model Predictive Control , 2020, IFAC-PapersOnLine.

[26]  Murat Arcak,et al.  Interval Reachability Analysis , 2021 .

[27]  D. Ruppert.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction , 2004 .

[28]  S. R. Searle,et al.  On Deriving the Inverse of a Sum of Matrices , 1981 .

[29]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[30]  Fabrizio Dabbene,et al.  Safe approximations of chance constrained sets by probabilistic scaling , 2019, 2019 18th European Control Conference (ECC).