Generalized Least Squares, Taylor Series Linearization and Fisher's Scoring in Multivariate Nonlinear Regression

In this article, we consider a general multivariate nonlinear regression setting in which the marginal mean and variance–covariance structure share a common set of regression parameters. Estimation is carried out via iteratively reweighted generalized least squares (IRGLS) that entails repeated application of Taylor series linearization and estimated generalized least squares (EGLS). Under normality, this IRGLS procedure is equivalent to Fisher's method of scoring and hence maximum likelihood estimation (MLE). However, estimates from this procedure are also shown to minimize a bias-corrected generalized least squares objective function that does not require the assumption of normality. Under fairly mild regularity conditions, the resulting estimates are consistent, asymptotically normal, and–under normality assumptions–asymptotically efficient. The estimates are compared against those obtained as solutions to the usual generalized estimating equations (GEE) using both simulation and numerical examples.

[1]  L B Sheiner,et al.  Estimating population kinetics. , 1982, Critical reviews in biomedical engineering.

[2]  P. Albert,et al.  Models for longitudinal data: a generalized estimating equation approach. , 1988, Biometrics.

[3]  Richard F. Gunst,et al.  Applied Regression Analysis , 1999, Technometrics.

[4]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[5]  D. Bates,et al.  Nonlinear mixed effects models for repeated measures data. , 1990, Biometrics.

[6]  D. Harville.  Matrix Algebra From a Statistician's Perspective , 1998 .

[7]  Martin Crowder,et al.  On the use of a working correlation matrix in using generalised linear models for repeated measures , 1995 .

[8]  E. Vonesh,et al.  Mixed-effects nonlinear regression for unbalanced repeated measures. , 1992, Biometrics.

[9]  A. Gallant,et al.  Nonlinear Statistical Models , 1988 .

[10]  Kalyan Das,et al.  Miscellanea. On the efficiency of regression estimators in generalised linear models for longitudinal data , 1999 .

[11]  S. Zeger,et al.  Longitudinal data analysis using generalized linear models , 1986 .

[12]  S. R. Searle,et al.  Vec and vech operators for matrices, with some uses in jacobians and multivariate statistics , 1979 .

[13]  D. A. Ratkowsky,et al.  Taylor Series Linearization and Scoring for Parameters in Nonlinear Regression , 1975 .

[14]  On Joint Estimation of Regression and Overdispersion Parameters in Generalized Linear Models for Longitudinal Data , 1996 .

[15]  Lewis B. Sheiner,et al.  Heteroscedastic nonlinear regression , 1988 .

[16]  S. R. Searle,et al.  The Vec-Permutation Matrix, the Vec Operator and Kronecker Products: A Review , 1981 .

[17]  D. Dockery,et al.  Passive smoking, gas cooking, and respiratory health of children living in six cities. , 2015, The American review of respiratory disease.

[18]  P. McCullagh,et al.  Generalized Linear Models , 1992 .

[19]  Thomas A. Severini,et al.  Extended Generalized Estimating Equations for Clustered Data , 1998 .

[20]  Abraham Charnes,et al.  The Equivalence of Generalized Least Squares and Maximum Likelihood Estimates in the Exponential Family , 1976 .

[21]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[22]  田中 豊.  Multivariate Statistical Modelling Based on Generalized Linear Models/Ludwig Fahrmeir,Gerhard Tutz(1994) , 1995 .

[23]  Calyampudi R. Rao,et al.  Linear Statistical Inference and Its Applications. , 1975 .

[24]  L. Fahrmeir,et al.  Multivariate statistical modelling based on generalized linear models , 1994 .

[25]  Eric R. Ziegel,et al.  Multivariate Statistical Modelling Based on Generalized Linear Models , 2002, Technometrics.

[26]  R. Prentice,et al.  Correlated binary regression with covariates specific to each binary observation. , 1988, Biometrics.

[27]  H. Goldstein.  Multilevel mixed linear model analysis using iterative generalized least squares , 1986 .

[28]  L. Skovgaard.  NONLINEAR MODELS FOR REPEATED MEASUREMENT DATA. , 1996 .

[29]  Edwin L. Bradley,et al.  The Equivalence of Maximum Likelihood and Weighted Least Squares Estimates in the Exponential Family , 1973 .

[30]  E. Vonesh,et al.  Linear and Nonlinear Models for the Analysis of Repeated Measurements , 1996 .

[31]  C. R. Rao,et al.  Linear Statistical Inference and its Applications , 1968 .

[32]  Norman R. Draper,et al.  Applied regression analysis (2. ed.) , 1981, Wiley series in probability and mathematical statistics.

[33]  L. Zhao,et al.  Estimating equations for parameters in means and covariances of multivariate discrete and continuous responses. , 1991, Biometrics.