Reduced-rank vector generalized linear models

Reduced-rank regression is a method with great potential for dimension reduction but has found few applications in applied statistics. To address this, reduced-rank regression is proposed for the class of vector generalized linear models (VGLMs), which is very large. The resulting class, which we call reduced-rank VGLMs (RR-VGLMs), enables the benefits of reduced-rank regression to be conveyed to a wide range of data types, including categorical data. RR-VGLMs are illustrated by focussing on models for categorical data, and especially the multinomial logit model. General algorithmic details are provided and software written by the first author is described. The reduced-rank multinomial logit model is illustrated with real data in two contexts: a regression analysis of workforce data and a classification problem.

[1]  C.J.F. ter Braak,et al.  A Theory of Gradient Analysis , 2004 .

[2]  T. W. Anderson.  Correction to "Estimating Linear Restrictions on Regression Coefficients for Multivariate Normal Distributions" , 1980 .

[3]  S. Zamir,et al.  Lower Rank Approximation of Matrices by Least Squares With Any Choice of Weights , 1979 .

[4]  S. Weisberg.  Dimension Reduction Regression in R , 2002 .

[5]  Ananda Sen,et al.  The Theory of Dispersion Models , 1997, Technometrics.

[6]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[7]  Leo A. Goodman,et al.  Association Models and Canonical Correlation in the Analysis of Cross-Classifications Having Ordered Categories , 1981 .

[8]  C.J.F. ter Braak,et al.  Biplots in Reduced-Rank Regression , 1994 .

[9]  C. Wild,et al.  Vector Generalized Additive Models , 1996 .

[10]  Robin Thompson,et al.  Composite Link Functions in Generalized Linear Models , 1981 .

[11]  Shelby J. Haberman,et al.  Tests for Independence in Two-Way Contingency Tables Based on Canonical Correlation and on Linear-By-Linear Interaction , 1981 .

[12]  W. Li,et al.  Time series models based on generalized linear models: some further results. , 1994, Biometrics.

[13]  Gordon K. Smyth,et al.  Partitioned algorithms for maximum likelihood and other non-linear estimation , 1996, Stat. Comput..

[14]  P. Green.  Iteratively reweighted least squares for maximum likelihood estimation , 1984 .

[15]  F. Richards.  A Method of Maximum‐Likelihood Estimation , 1961 .

[16]  Gregory C. Reinsel,et al.  Nested Reduced-Rank Autoregressive Models for Multiple Time Series , 1988 .

[17]  G. Reinsel,et al.  Multivariate Reduced-Rank Regression: Theory and Applications , 1998 .

[18]  S Greenland,et al.  Alternative models for ordinal logistic regression. , 1994, Statistics in medicine.

[19]  H. O. Hartley,et al.  Biometrika Tables for Statisticians, Vol. 2 , 1973 .

[20]  Jean-Baptiste Denis,et al.  Asymptotic Confidence Regions for Biadditive Models: Interpreting Genotype‐Environment Interactions , 1996 .

[21]  Mikis D. Stasinopoulos,et al.  Fitting Non-Gaussian Time Series Models , 1998, COMPSTAT.

[22]  John Hinde,et al.  Statistical Modelling in GLIM. , 1989 .

[23]  H. Akaike.  A new look at the statistical model identification , 1974 .

[24]  T. W. Anderson.  Estimating Linear Restrictions on Regression Coefficients for Multivariate Normal Distributions , 1951 .

[25]  C. Eckart,et al.  The approximation of one matrix by another of lower rank , 1936 .

[26]  Heinz Schmidli,et al.  Reduced Rank Regression , 1995 .

[27]  P. McCullagh.  Regression Models for Ordinal Data , 1980 .

[28]  J. Anderson.  Regression and Ordered Categorical Variables , 1984 .

[29]  B. Francis,et al.  Fitting bilinear models in GLIM. , 1995 .

[30]  J. Friedman,et al.  Projection Pursuit Regression , 1981 .

[31]  P. McCullagh,et al.  Generalized Linear Models, 2nd Edn. , 1990 .

[32]  R. Tibshirani,et al.  Flexible Discriminant Analysis by Optimal Scoring , 1994 .

[33]  A. Izenman.  Reduced-rank regression for the multivariate linear model , 1975 .

[34]  K. Gabriel,et al.  Generalised bilinear regression , 1998 .

[35]  Antoine Guisan,et al.  Ordinal response regression models in ecology. , 2000 .