A Localized Neural Network with Dependent Data: Estimation and Inference

In this paper, we propose a localized neural network (LNN) model and then develop the LNN based estimation and inferential procedures for dependent data in both cases with quantitative/qualitative outcomes. We explore the use of identification restrictions from a nonparametric regression perspective, and establish an estimation theory for the LNN setting under a set of mild conditions. The asymptotic distributions are derived accordingly, and we show that LNN automatically eliminates the dependence of data when calculating the asymptotic variances. The finding is important, as one can easily use different types of wild bootstrap methods to obtain valid inference practically. In particular, for quantitative outcomes, the proposed LNN approach yields closed-form expressions for the estimates of some key estimators of interest. Last but not least, we examine our theoretical findings through extensive numerical studies.

[1]  Jianqing Fan,et al.  Factor Augmented Sparse Throughput Deep ReLU Neural Networks for High Dimensional Regression , 2022, 2210.02002.

[2]  Patrick Vossler,et al.  Dimension-Free Average Treatment Effect Inference with Deep Neural Networks , 2021, ArXiv.

[3]  B. Chaudhuri,et al.  Activation functions in deep learning: A comprehensive survey and benchmark , 2021, Neurocomputing.

[4]  Wei Lin,et al.  Harmless Overparametrization in Two-layer Neural Networks , 2021, ArXiv.

[5]  Andrea Montanari,et al.  Deep learning: a statistical viewpoint , 2021, Acta Numerica.

[6]  Xiaohong Chen,et al.  Causal Inference of General Treatment Effects using Neural Networks with A Diverging Number of Confounders , 2020, 2009.07055.

[7]  Bryan T. Kelly,et al.  Autoencoder Asset Pricing Models , 2019, Journal of Econometrics.

[8]  M. Kohler,et al.  On deep learning as a remedy for the curse of dimensionality in nonparametric regression , 2019, The Annals of Statistics.

[9]  Cong Ma,et al.  A Selective Overview of Deep Learning , 2019, Statistical science : a review journal of the Institute of Mathematical Statistics.

[10]  Manfred Mudelsee,et al.  Trend analysis of climate time series: A review of methods , 2019, Earth-Science Reviews.

[11]  Bryan T. Kelly,et al.  Empirical Asset Pricing Via Machine Learning , 2018, The Review of Financial Studies.

[12]  Sanjog Misra,et al.  Deep Neural Networks for Estimation and Inference , 2018, Econometrica.

[13]  Susan Athey,et al.  The Impact of Machine Learning on Economics , 2018, The Economics of Artificial Intelligence.

[14]  Johannes Schmidt-Hieber,et al.  Nonparametric regression using deep neural networks with ReLU activation function , 2017, The Annals of Statistics.

[15]  Robert F. Engle,et al.  Large Dynamic Covariance Matrices , 2017 .

[16]  Adam Krzyżak,et al.  Nonparametric Regression Based on Hierarchical Interaction Models , 2017, IEEE Transactions on Information Theory.

[17]  D. Tjøstheim,et al.  Estimation in nonlinear regression with Harris recurrent Markov chains , 2016, 1609.04237.

[18]  Kevin Sheppard,et al.  Good Volatility, Bad Volatility: Signed Jumps and The Persistence of Volatility , 2013, Review of Economics and Statistics.

[19]  Patrick M. Kline,et al.  A Score Based Approach to Wild Bootstrap Inference , 2010 .

[20]  J. Horowitz,et al.  Asymptotic properties of bridge estimators in sparse high-dimensional regression models , 2008, 0804.0693.

[21]  Philippe Jorion,et al.  Firm Value and Hedging: Evidence from U.S. Oil and Gas Producers , 2006 .

[22]  Bill Nixon,et al.  Evidence on the nature and extent of the relationship between oil prices and equity values in the UK , 2005 .

[23]  Francis X. Diebold,et al.  The Rodney L. White Center for Financial Research Financial Asset Returns, Direction-of-Change Forecasting and Volatility , 2003 .

[24]  W. Newey,et al.  Instrumental variable estimation of nonparametric models , 2003 .

[25]  Jeffrey S. Racine,et al.  Semiparametric ARX neural-network models with an application to forecasting inflation , 2001, IEEE Trans. Neural Networks.

[26]  Halbert White,et al.  Improved Rates and Asymptotic Normality for Nonparametric Neural Network Estimators , 1999, IEEE Trans. Inf. Theory.

[27]  Xiaotong Shen,et al.  Sieve extremum estimates for weakly dependent data , 1998 .

[28]  William Remus,et al.  Neural Network Models for Time Series Forecasts , 1996 .

[29]  Shun-ichi Amari,et al.  Network information criterion-determining the number of hidden units for an artificial neural network model , 1994, IEEE Trans. Neural Networks.

[30]  Ali A. Minai,et al.  On the derivatives of the sigmoid , 1993, Neural Networks.

[31]  George Cybenko,et al.  Approximation by superpositions of a sigmoidal function , 1989, Math. Control. Signals Syst..

[32]  Ing,et al.  Deep learning for the partially linear Cox model , 2022, The Annals of Statistics.

[33]  Kathleen B. Cooper,et al.  Introduction (Extract 1 from The Economics of Artificial Intelligence: An Agenda) , 2019 .

[34]  Thorsten Gerber,et al.  Handbook Of Mathematical Functions , 2016 .

[35]  Stefan Fritsch,et al.  neuralnet: Training of Neural Networks , 2010, R J..

[36]  Xiaohong Chen.  Chapter 76 Large Sample Sieve Estimation of Semi-Nonparametric Models , 2007 .

[37]  Tomas Sauer,et al.  Polynomial Interpolation in Several Variables: Lattices, Differences, and Ideals , 2006 .

[38]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[39]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[40]  J. Magnus,et al.  Matrix Differential Calculus with Applications , 1988 .