Partially Linear Models

In the last ten years, there has been increasing interest and activity in the general area of partially linear regression smoothing in statistics. Many methods and techniques have been proposed and studied. This monograph hopes to bring an up-to-date presentation of the state of the art of partially linear regression techniques. The emphasis of this monograph is on methodologies rather than on the theory, with a particular focus on applications of partially linear regression techniques to various statistical problems. These problems include least squares regression, asymptotically efficient estimation, bootstrap resampling, censored data analysis, linear measurement error models, nonlinear measurement models, nonlinear and nonparametric time series models. We hope that this monograph will serve as a useful reference for theoretical and applied statisticians and to graduate students and others who are interested in the area of partially linear regression. While advanced mathematical ideas have been valuable in some of the theoretical development, the methodological power of partially linear regression can be demonstrated and discussed without advanced mathematics. This monograph can be divided into three parts: part one–Chapter 1 through Chapter 4; part two–Chapter 5; and part three–Chapter 6. In the first part, we discuss various estimators for partially linear regression models, establish theo- retical results for the estimators, propose estimation procedures, and implement the proposed estimation procedures through real and simulated examples. The second part is of more theoretical interest. In this part, we construct several adaptive and efficient estimates for the parametric component. We show that the LS estimator of the parametric component can be modified to have both Bahadur asymptotic efficiency and second order asymptotic efficiency. In the third part, we consider partially linear time series models. First, we propose a test procedure to determine whether a partially linear model can be used to fit a given set of data. Asymptotic test criteria and power investigations are presented. Second, we propose a Cross-Validation (CV) based criterion to select the optimum linear subset from a partially linear regression and estab- lish a CV selection criterion for the bandwidth involved in the nonparametric kernel estimation. The CV selection criterion can be applied to the case where the observations fitted by the partially linear model (1.1.1) are independent and identically distributed (i.i.d.). Due to this reason, we have not provided a sepa- rate chapter to discuss the selection problem for the i.i.d. case. Third, we provide recent developments in nonparametric and semiparametric time series regression. This work of the authors was supported partially by the Sonderforschungs- bereich373“QuantifikationundSimulationO konomischerProzesse”.Thesecond author was also supported by the National Natural Science Foundation of China and an Alexander von Humboldt Fellowship at the Humboldt University, while the third author was also supported by the Australian Research Council. The second and third authors would like to thank their teachers: Professors Raymond Car- roll, Guijing Chen, Xiru Chen, Ping Cheng and Lincheng Zhao for their valuable inspiration on the two authors’ research efforts. We would like to express our sin- cere thanks to our colleagues and collaborators for many helpful discussions and stimulating collaborations, in particular, Vo Anh, Shengyan Hong, Enno Mam- men, Howell Tong, Axel Werwatz and Rodney Wolff. For various ways in which they helped us, we would like to thank Adrian Baddeley, Rong Chen, Anthony Pettitt, Maxwell King, Michael Schimek, George Seber, Alastair Scott, Naisyin Wang, Qiwei Yao, Lijian Yang and Lixing Zhu. The authors are grateful to everyone who has encouraged and supported us to finish this undertaking. Any remaining errors are ours.

[1]  D. Tjøstheim,et al.  Nonparametric Estimation and Identification of Nonlinear ARCH Time Series Strong Convergence and Asymptotic Normality: Strong Convergence and Asymptotic Normality , 1995, Econometric Theory.

[2]  Jianqing Fan,et al.  Nonparametric regression with errors in variables , 1993 .

[3]  Leon Jay Gleser,et al.  The Importance of Assessing Measurement Reliability in Multivariate Regression , 1992 .

[4]  Jiti Gao,et al.  The laws of the iterated logarithm of some estimates in partly linear models , 1995 .

[5]  Nancy E. Heckman,et al.  Spline Smoothing in a Partly Linear Model , 1986 .

[6]  P. Robinson.  ROOT-N-CONSISTENT SEMIPARAMETRIC REGRESSION , 1988 .

[7]  Bernd Rönz,et al.  Credit Scoring using Semiparametric Methods , 2000 .

[8]  S. Hu.  THE STRONG UNIFORM CONSISTENCY OF KERNEL DENSITY ESTIMATES FOR φ—MIXING SAMPLE , 1993 .

[9]  Anton Schick,et al.  Weighted least squares estimates in partly linear regression models , 1996 .

[10]  Mai Zhou,et al.  Asymptotic Normality of the `Synthetic Data' Regression Estimator for Censored Survival Data , 1992 .

[11]  George E. P. Box,et al.  Correcting Inhomogeneity of Variance with Power Transformation Weighting , 1974 .

[12]  Philippe Vieu,et al.  Choice of regressors in nonparametric estimation , 1994 .

[13]  Hua Liang,et al.  Second-order asymptotic efficiency of PMLE in generalized linear models , 1995 .

[14]  Jiti Gao,et al.  Asymptotic theory for partly linear models , 1994 .

[15]  R. Tibshirani,et al.  Generalized additive models for medical research , 1986, Statistical methods in medical research.

[16]  A. Gallant,et al.  Finding Chaos in Noisy Systems , 1992 .

[17]  Raymond J. Carroll,et al.  Variance Function Estimation in Regression: the Effect of Estimating the Mean , 1988 .

[18]  Robert J. Willis,et al.  Chapter 10 Wage determinants: A survey and reinterpretation of human capital earnings functions , 1986 .

[19]  W. Härdle,et al.  Estimation in a semiparametric partially linear errors-in-variables model , 1999 .

[20]  Raymond J. Carroll,et al.  Adapting for Heteroscedasticity in Linear Models , 1982 .

[21]  Anton Schick,et al.  Root-n consistent estimation in partly linear regression models☆ , 1996 .

[22]  Qiwei Yao,et al.  On subset selection in non-parametric stochastic regression , 1994 .

[23]  Yanqin Fan,et al.  Consistent model specification tests : Omitted variables and semiparametric functional forms , 1996 .

[24]  J. N. R. Jeffers,et al.  "Fitting Equations to Data: Computer Analysis of Multifactor Data for Scientists and Engineers, " by C. Daniel and F. S. Wood (Book Review) , 1974, Int. J. Man Mach. Stud..

[25]  J. Rice.  Convergence rates for partially splined models , 1986 .

[26]  L. V. Osipov.  Asymptotic Expansions of the Distribution Function of a Sum of Independent Lattice Random Variables , 1969 .

[27]  Liang Hua.  Asymptotic efficiency in semiparametric models , 1996 .

[28]  Tak K. Mak,et al.  Estimation of Parameters in Heteroscedastic Linear Models , 1992 .

[29]  W. Härdle,et al.  A Review of Nonparametric Time Series Analysis , 1997 .

[30]  Wayne A. Fuller,et al.  Measurement Error Models , 1988 .

[31]  Liang Hua.  ON THE SMALLEST POSSIBLE ASYMPTOTICALLY EFFICIENT VARIANCE IN SEMIPARAMETRIC MODELS , 1994 .

[32]  赤平 昌文,et al.  Asymptotic efficiency of statistical estimators : concepts and higher order asymptotic efficiency , 1981 .

[33]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[34]  H. Tong.  A Personal Overview Of Nonlinear Time-Series Analysis From A Chaos Perspective , 1995 .

[35]  Hung Chen,et al.  Convergence Rates for Parametric Components in a Partly Linear Model , 1988 .

[36]  B. Yandell,et al.  Semi-Parametric Generalized Linear Models. , 1985 .

[37]  Hua Liang,et al.  Asymptotic normality in a Semiparametric partially linear model with right–censored data , 1998 .

[38]  Trevor Hastie,et al.  Statistical Models in S , 1991 .

[39]  Howell Tong,et al.  Nonparametric function estimation in noisy chaos , 1993 .

[40]  Hua Liang.  The berry-esseen bounds of error variance estimation in semiparametric regression models , 1994 .

[41]  W. Härdle,et al.  XploRe Learning Guide , 1999 .

[42]  A. A. Weiss,et al.  Semiparametric estimates of the relation between weather and electricity sales , 1986 .

[43]  Michael Buckley,et al.  AN APPROXIMATION TO THE DISTRIBUTION OF QUADRATIC FORMS IN NORMAL RANDOM VARIABLES , 1988 .

[44]  Young K. Truong,et al.  Local Linear Estimation in Partly Linear Models , 1997 .

[45]  Richard Schmalensee,et al.  Household Gasoline Demand in the United States , 1999 .

[46]  J. Fu,et al.  On a Theorem of Bahadur on the Rate of Convergence of Point Estimators , 1973 .

[47]  Clifford H. Spiegelman,et al.  Testing the Goodness of Fit of a Linear Model via Nonparametric Regression Techniques , 1990 .

[48]  Graciela Boente,et al.  Consistency of a nonparametric estimate of a density function for dependent variables , 1988 .

[49]  Hua Liang,et al.  On Bahadur asymptotic efficiency in a semiparametric regression model , 1994 .

[50]  Oliver Linton,et al.  SECOND ORDER APPROXIMATION IN THE PARTIALLY LINEAR REGRESSION MODEL , 1995 .

[51]  Raymond J. Carroll,et al.  Second order effects in semiparametric weighted least squares regression , 1988 .

[52]  Jiti Gao,et al.  M-TYPE SMOOTHING SPLINES IN NONPARAMETRIC AND SEMIPARAMETRIC REGRESSION MODELS , 1997 .

[53]  Carlo Novara,et al.  of nonlinear time series , 2001 .

[54]  A. Oswald,et al.  The Wage Curve , 1989 .

[55]  Howell Tong.  Some comments on the Canadian Lynx data with discussion , 1977 .

[56]  Jun S. Liu,et al.  Additivity tests for nonlinear autoregression , 1995 .

[57]  A. Schick.  On Asymptotically Efficient Estimation in Semiparametric Models , 1986 .

[58]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[59]  Sara van de Geer,et al.  Penalized quasi-likelihood estimation in partial linear models , 1997 .

[60]  C. J. Stone.  OPTIMAL GLOBAL RATES OF CONVERGENCE FOR NONPARAMETRIC ESTIMATORS , 1982 .

[61]  C. J. Stone,et al.  Additive Regression and Other Nonparametric Models , 1985 .

[62]  Joan G. Staniswalis.  Correction to Severini and Staniswalis (1994): Quasi-Likelihood Estimation in Semiparametric Models , 1994 .

[63]  R. Carroll,et al.  Deconvolving kernel density estimators , 1987 .

[64]  Michael G. Schimek,et al.  Estimation in partially linear models , 1998 .

[65]  Enno Mammen,et al.  Testing Parametric Versus Semiparametric Modelling in Generalized Linear Models , 1996 .

[66]  Dag Tjøstheim,et al.  Nonparametric Identification of Nonlinear Time Series: Projections , 1994 .

[67]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[68]  Ruey S. Tsay,et al.  Nonlinear Additive ARX Models , 1993 .

[69]  M. C. Jones,et al.  Spline Smoothing and Nonparametric Regression. , 1989 .

[70]  From Clocks to Chaos: The Rhythms of Life , 1988 .

[71]  Jiti Gao,et al.  Adaptive orthogonal series estimation in additive stochastic regression models , 2002 .

[72]  Wolfgang Härdle,et al.  Nonparametric Curve Estimation from Time Series , 1989 .

[73]  Anton Schick,et al.  On efficient estimation in regression models , 1993 .

[74]  Hua Liang,et al.  Statistical Inference in Single-Index and Partially Nonlinear Models , 1997 .

[75]  P. Anglin,et al.  SEMIPARAMETRIC ESTIMATION OF A HEDONIC PRICE FUNCTION , 1996 .

[76]  Gregg E. Dinse,et al.  Correction: Regression Analysis of Tumour Prevalence Data , 1984 .

[77]  Jack Cuzick,et al.  Efficient Estimates in Semiparametric Additive Regression Models with Unknown Error Distribution , 1992 .

[78]  B. Silverman,et al.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[79]  S Hong.  ESTIMATION THEORY OF A CLASS OF SEMIPARAMETRIC REGRESSION MODELS , 1992 .

[80]  D. Andrews.  Asymptotic Normality of Series Estimators for Nonparametric and Semiparametric Regression Models , 1991 .

[81]  P. Bickel.  On Adaptive Estimation , 1982 .

[82]  Ulrich Rendtel,et al.  Zum Zusammenhang zwischen Lohnhöhe und Arbeitslosigkeit: Neue Befunde auf Basis semi-parametrischer Schätzungen und eines verallgemeinerten Varianz-Komponenten-Modells , 1995 .

[83]  A. Seheult,et al.  Analysis of Field Experiments by Least Squares Smoothing , 1985 .

[84]  D. Pollard.  Convergence of stochastic processes , 1984 .

[85]  Timo Teräsvirta,et al.  Aspects of modelling nonlinear time series , 1986 .

[86]  Michael G. Schimek.  Non- and Semiparametric Alternatives to Generalized Linear Models , 1997 .

[87]  Steven A. Orszag,et al.  CBMS-NSF REGIONAL CONFERENCE SERIES IN APPLIED MATHEMATICS , 1978 .

[88]  G. Wahba.  Spline models for observational data , 1990 .

[89]  Bodhini R. Jayasuriya.  Testing for Polynomial Regression Using Nonparametric Regression Techniques , 1996 .

[90]  Vo Anh,et al.  Semiparametric regression under long-range dependent errors , 1999 .

[91]  Ulrich Stadtmüller,et al.  Estimation of Heteroscedasticity in Regression Analysis , 1987 .

[92]  A. Bowman,et al.  Applied smoothing techniques for data analysis : the kernel approach with S-plus illustrations , 1999 .

[93]  Changbao Wu,et al.  Jackknife, Bootstrap and Other Resampling Methods in Regression Analysis , 1986 .

[94]  Wayne A. Fuller,et al.  Least Squares Estimation When the Covariance Matrix and Parameter Vector are Functionally Related , 1980 .

[95]  W. J. Hall,et al.  Information and Asymptotic Efficiency in Parametric-Nonparametric Models , 1983 .

[96]  W. Fuller,et al.  Estimation for a Linear Regression Model with Unknown Diagonal Covariance Matrix , 1978 .

[97]  P. Bickel,et al.  Using Residuals Robustly I: Tests for Heteroscedasticity, Nonlinearity , 1975 .

[98]  Michael G. Schimek,et al.  Estimation and inference in partially linear models with smoothing splines , 2000 .

[99]  Dag Tjøstheim,et al.  Additive Nonlinear ARX Time Series and Projection Estimates , 1997, Econometric Theory.

[100]  P. K. Bhattacharya,et al.  Semiparametric inference in a partial linear model , 1997 .

[101]  George G. Lorentz,et al.  Constructive Approximation , 1993, Grundlehren der mathematischen Wissenschaften.

[102]  Jianqing Fan,et al.  Generalized Partially Linear Single-Index Models , 1997 .

[103]  P. Hall,et al.  Martingale Limit Theory and its Application. , 1984 .

[104]  K. Bellmann.  Daniel, C., F. S. WOOD, J. W. GORMAN: Fitting Equations to Data. Computer Analysis of Multifactor Data for Scientists and Engineers. John Wiley & Sons, New York-London-Sydney-Toronto 1974. XIV, 342 S., 132 Abb., 33 Tab., £6.50 , 1975 .

[105]  Jack Cuzick,et al.  Semiparametric additive regression , 1992 .

[106]  Wolfgang Härdle,et al.  On adaptive estimation in partial linear models , 2000 .

[107]  Jiti Gao,et al.  Adaptive Estimation in Partly Linear Regression Models , 1993 .

[108]  T. Lai,et al.  Functional Laws of the Iterated Logarithm for the Product-Limit Estimator of a Distribution Function Under Random Censorship or Truncation , 1990 .

[109]  Dag Tjøstheim,et al.  Nonparametric tests of linearity for time series , 1995 .

[110]  A. Gallant,et al.  On the bias in flexible functional forms and an essentially unbiased form : The fourier flexible form , 1981 .

[111]  I. James,et al.  Linear regression with censored data , 1979 .

[112]  Zhiliang Ying,et al.  Rank Regression Methods for Left-Truncated and Right-Censored Data , 1991 .

[113]  Hua Liang,et al.  Large sample theory of the estimation of the error distribution for a semiparametric model , 1997 .

[114]  J. V. Ryzin,et al.  Regression Analysis with Randomly Right-Censored Data , 1981 .

[115]  W. Stout.  Almost sure convergence , 1974 .

[116]  Gautam Tripathi,et al.  LOCAL SEMIPARAMETRIC EFFICIENCY BOUNDS UNDER SHAPE RESTRICTIONS , 2000, Econometric Theory.

[117]  Hua Liang,et al.  Asymptotic normality of pseudo-LS estimator for partly linear autoregression models , 1995 .

[118]  P. Speckman.  Kernel smoothing in partial linear models , 1988 .

[119]  Chai Gen.  Asymptotic Theory for Estimation of Error Distribution in Linear Model , 1993 .

[120]  C. J. Stone,et al.  Adaptive Maximum Likelihood Estimators of a Location Parameter , 1975 .

[121]  David Ruppert,et al.  Robust Estimation in Heteroscedastic Linear Models. , 1982 .

[122]  Michel Loève,et al.  Probability Theory I , 1977 .

[123]  E. Mammen,et al.  Comparing Nonparametric Versus Parametric Regression Fits , 1993 .

[124]  H. Tong,et al.  On consistent nonparametric order determination and chaos , 1992 .

[125]  Robert Kohn,et al.  A BAYESIAN APPROACH TO ESTIMATING AND FORECASTING ADDITIVE NONPARAMETRIC AUTOREGRESSIVE MODELS , 1996 .

[126]  Anthony C. Davison,et al.  Bootstrap Methods and Their Application , 1998 .

[127]  Hung Chen,et al.  Selection of the splined variables and convergence rates in a partial spline model , 1991 .

[128]  R. R. Bahadur.  Rates of Convergence of Estimates and Test Statistics , 1967 .

[129]  Randall Eubank,et al.  Convergence rates for estimation in certain partially linear models , 1989 .

[130]  Wolfgang Härdle,et al.  Applied Nonparametric Regression , 1991 .

[131]  Liang Hua,et al.  Second order asymptotic efficiency in a partial linear model , 1993 .

[132]  K. Do,et al.  Efficient and Adaptive Estimation for Semiparametric Models. , 1994 .

[133]  H. An,et al.  The geometrical ergodicity of nonlinear autoregressive models , 1996 .

[134]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[135]  D. Tjøstheim.  Non-linear Time Series: A Selective Review* , 1994 .

[136]  J. W. Gorman,et al.  Fitting Equations to Data. , 1973 .

[137]  Gregg E. Dinse,et al.  Regression Analysis of Tumour Prevalence Data , 1983 .

[138]  D. Cox.  Nonparametric Regression and Generalized Linear Models: A roughness penalty approach , 1993 .

[139]  P. Doukhan.  Mixing: Properties and Examples , 1994 .

[140]  T. Severini,et al.  Quasi-Likelihood Estimation in Semiparametric Models , 1994 .

[141]  W. Härdle,et al.  Kernel regression smoothing of time series , 1992 .

[142]  Yongmiao Hong,et al.  Consistent Specification Testing via Nonparametric Series Regression , 1995 .

[143]  K. Takeuchi,et al.  Asymptotic Efficiency of Statistical Estimators: Concepts and Higher Order Asymptotic Efficiency (Lecture Notes in Statistics, Vol. 7). , 1982 .

[144]  Dag Tjøstheim,et al.  Nonparametric Identification of Nonlinear Time Series: Selecting Significant Lags , 1994 .

[145]  Jiti Gao,et al.  Semiparametric Regression Smoothing of Non‐linear Time Series , 1998 .