Automatic Inference of the Quantile Parameter

Supervised learning is an active research area, with numerous applications in diverse fields such as data analytics, computer vision, speech and audio processing, and image understanding. In most cases, the loss functions used in machine learning assume symmetric noise models, and seek to estimate the unknown function parameters. However, loss functions such as quantile and quantile Huber generalize the symmetric $\ell_1$ and Huber losses to the asymmetric setting, for a fixed quantile parameter. In this paper, we propose to jointly infer the quantile parameter and the unknown function parameters, for the asymmetric quantile Huber and quantile losses. We explore various properties of the quantile Huber loss and implement a convexity certificate that can be used to check convexity in the quantile parameter. When the loss if convex with respect to the parameter of the function, we prove that it is biconvex in both the function and the quantile parameters, and propose an algorithm to jointly estimate these. Results with synthetic and real data demonstrate that the proposed approach can automatically recover the quantile parameter corresponding to the noise and also provide an improved recovery of function parameters. To illustrate the potential of the framework, we extend the gradient boosting machines with quantile losses to automatically estimate the quantile parameter at each iteration.

[1]  G. Pillonetto,et al.  An $\ell _{1}$-Laplace Robust Kalman Smoother , 2011, IEEE Transactions on Automatic Control.

[2]  R. Tyrrell Rockafellar,et al.  Variational Analysis , 1998, Grundlehren der mathematischen Wissenschaften.

[3]  Hui Zou,et al.  Computational Statistics and Data Analysis Regularized Simultaneous Model Selection in Multiple Quantiles Regression , 2022 .

[4]  Songfeng Zheng,et al.  QBoost: Predicting quantiles with boosting for regression and binary classification , 2012, Expert Syst. Appl..

[5]  Stephen P. Boyd,et al.  Smoothed state estimates under abrupt changes using sum-of-norms regularization , 2012, Autom..

[6]  Moshe Buchinsky CHANGES IN THE U.S. WAGE STRUCTURE 1963-1987: APPLICATION OF QUANTILE REGRESSION , 1994 .

[7]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[8]  R. Koenker Quantile Regression: Fundamentals of Quantile Regression , 2005 .

[9]  Junbin Gao,et al.  Robust L1 Principal Component Analysis and Its Bayesian Variational Inference , 2008, Neural Computation.

[10]  Aleksandr Y. Aravkin,et al.  Estimating nuisance parameters in inverse problems , 2012, 1206.6532.

[11]  R. Koenker,et al.  Reappraising Medfly Longevity , 2001 .

[12]  R. Koenker,et al.  Regression Quantiles , 2007 .

[13]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[14]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[15]  Ronny Luss,et al.  Orthogonal Matching Pursuit for Sparse Quantile Regression , 2014, 2014 IEEE International Conference on Data Mining.

[16]  Georgios B. Giannakis,et al.  Doubly Robust Smoothing of Dynamical Processes via Outlier Sparsity Constraints , 2011, IEEE Transactions on Signal Processing.

[17]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[18]  J. Friedman Greedy function approximation: A gradient boosting machine. , 2001 .