Adaptive quantile regressions for massive datasets

Analysis of massive datasets is challenging owing to limitations of computer primary memory. Adaptive quantile regressions is a robust and efficient estimation method. For computational efficiency, we propose an adaptive smoothing quantile regressions (ASQR). The ASQR method is used to analyze massive datasets. The proposed approach significantly reduces the required amount of primary memory, and the resulting estimate will be as efficient as if the entire data set is analyzed simultaneously. Both simulations and data analysis are conducted to illustrate the finite sample performance of the proposed methods.

[1]  Robust and efficient estimation with weighted composite quantile regression , 2016 .

[2]  R. Koenker Quantile Regression: Name Index , 2005 .

[3]  G. Claeskens,et al.  Composite versus model-averaged quantile regression , 2019, Journal of Statistical Planning and Inference.

[4]  N. Draper,et al.  Applied Regression Analysis: Draper/Applied Regression Analysis , 1998 .

[5]  Ruibin Xi,et al.  Aggregated estimating equation estimation , 2011 .

[6]  Qifa Xu,et al.  Block average quantile regression for massive dataset , 2017, Statistical Papers.

[7]  Tsai-Hung Fan,et al.  Regression analysis for massive datasets , 2007, Data Knowl. Eng..

[8]  Ke Yang,et al.  Adaptive composite quantile regressions and their asymptotic relative efficiency , 2018 .

[9]  Minge Xie,et al.  A Split-and-Conquer Approach for Analysis of Extraordinarily Large Data , 2014 .

[10]  N. Draper,et al.  Applied Regression Analysis , 1967 .

[11]  Heng Lian,et al.  A note on the efficiency of composite quantile regression , 2016 .

[12]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[13]  Xi Chen,et al.  Quantile regression under memory constraint , 2018, The Annals of Statistics.

[14]  J. Horowitz Bootstrap Methods for Median Regression Models , 1996 .

[15]  Roger Koenker,et al.  A note on L-estimates for linear models , 1984 .

[16]  Keming Yu,et al.  Composite quantile regression for massive datasets , 2018, Statistics.

[17]  Jing Wu,et al.  Online Updating of Statistical Inference in the Big Data Setting , 2015, Technometrics.

[18]  Qianqian Zhu,et al.  Estimation of linear composite quantile regression using EM algorithm , 2016 .

[19]  Lei Pang,et al.  Variance estimation in censored quantile regression via induced smoothing , 2012, Comput. Stat. Data Anal..