A partitioned quasi-likelihood for distributed statistical inference

In the big data setting, working data sets are often distributed on multiple machines. However, classical statistical methods are often developed to solve the problems of single estimation or inference. We employ a novel parallel quasi-likelihood method in generalized linear models, to make the variances between different sub-estimators relatively similar. Estimates are obtained from projection subsets of data and later combined by suitably-chosen unknown weights. We also show the proposed method to produce better asymptotic efficiency than using the simple average. Furthermore, simulation examples show that the proposed method can significantly improve statistical inference.

[1]  Martin J. Wainwright,et al.  Iterative Hessian Sketch: Fast and Accurate Solution Approximation for Constrained Least-Squares , 2014, J. Mach. Learn. Res..

[2]  P. Deuflhard.  Newton Methods for Nonlinear Problems: Affine Invariance and Adaptive Algorithms , 2011 .

[3]  J. Matousek,et al.  On variants of the Johnson–Lindenstrauss lemma , 2008 .

[4]  L. Fahrmeir,et al.  Multivariate statistical modelling based on generalized linear models , 1994 .

[5]  Yun Yang,et al.  Communication-Efficient Distributed Statistical Inference , 2016, Journal of the American Statistical Association.

[6]  Jianqing Fan,et al.  Distributed Estimation and Inference with Statistical Guarantees , 2015, 1509.05457.

[7]  Wei Shao,et al.  Parallel maximum likelihood estimator for multiple linear regression models , 2015, J. Comput. Appl. Math..

[8]  R. Little,et al.  Penalized Spline of Propensity Methods for Treatment Comparison , 2019, Journal of the American Statistical Association.

[9]  Darren J. Wilkinson,et al.  Scalable inference for Markov processes with intractable likelihoods , 2014, Stat. Comput..

[10]  S. Lang.  Real and Functional Analysis , 1983 .

[11]  P. Deuflhard.  The Grand Four: Affine Invariant Globalizations of Newton’s Method , 2018, Vietnam Journal of Mathematics.

[12]  F. Liang,et al.  A split‐and‐merge Bayesian variable selection approach for ultrahigh dimensional regression , 2015 .

[13]  Cheng Huang,et al.  A distributed one-step estimator , 2015, Math. Program..

[14]  James R. Gattiker,et al.  Parallel Bayesian Additive Regression Trees , 2013, 1309.1906.

[15]  Nate Strawn,et al.  Distributed Statistical Estimation and Rates of Convergence in Normal Approximation , 2017, Electronic Journal of Statistics.

[16]  F. Carbonell,et al.  Multiple Shooting-Local Linearization method for the identification of dynamical systems , 2015, Commun. Nonlinear Sci. Numer. Simul..

[17]  Srijan Sengupta,et al.  A Subsampled Double Bootstrap for Massive Data , 2015, 1508.01126.

[18]  Purnamrita Sarkar,et al.  A scalable bootstrap for massive data , 2011, 1112.5016.

[19]  P. Deuflhard,et al.  Parameter Identification in a Tuberculosis Model for Cameroon , 2013, PloS one.

[20]  Lei Zhang,et al.  Real-Time Compressive Tracking , 2012, ECCV.

[21]  Yee Whye Teh,et al.  Distributed Bayesian Learning with Stochastic Natural Gradient Expectation Propagation and the Posterior Server , 2015, J. Mach. Learn. Res..

[22]  T. Hu,et al.  Stability for randomly weighted sums of random elements , 1999 .

[23]  Ohad Shamir,et al.  Communication-Efficient Distributed Optimization using an Approximate Newton-type Method , 2013, ICML.