The jackknife and bootstrap

1. Introduction.- 1.1 Statistics and Their Sampling Distributions.- 1.2 The Traditional Approach.- 1.3 The Jackknife.- 1.4 The Bootstrap.- 1.5 Extensions to Complex Problems.- 1.6 Scope of Our Studies.- 2. Theory for the Jackknife.- 2.1 Variance Estimation for Functions of Means.- 2.1.1 Consistency.- 2.1.2 Other properties.- 2.1.3 Discussions and examples.- 2.2 Variance Estimation for Functionals.- 2.2.1 Differentiability and consistency.- 2.2.2 Examples.- 2.2.3 Convergence rate.- 2.2.4 Other differential approaches.- 2.3 The Delete-d Jackknife.- 2.3.1 Variance estimation.- 2.3.2 Jackknife histograms.- 2.4 Other Applications.- 2.4.1 Bias estimation.- 2.4.2 Bias reduction.- 2.4.3 Miscellaneous results.- 2.5 Conclusions and Discussions.- 3. Theory for the Bootstrap.- 3.1 Techniques in Proving Consistency.- 3.1.1 Bootstrap distribution estimators.- 3.1.2 Mallows' distance.- 3.1.3 Berry-Esseen's inequality.- 3.1.4 Imitation.- 3.1.5 Linearization.- 3.1.6 Convergence in moments.- 3.2 Consistency: Some Major Results.- 3.2.1 Distribution estimators.- 3.2.2 Variance estimators.- 3.3 Accuracy and Asymptotic Comparisons.- 3.3.1 Convergence rate.- 3.3.2 Asymptotic minimaxity.- 3.3.3 Asymptotic mean squared error.- 3.3.4 Asymptotic relative error.- 3.3.5 Conclusions.- 3.4 Fixed Sample Performance.- 3.4.1 Moment estimators.- 3.4.2 Distribution estimators.- 3.4.3 Conclusions.- 3.5 Smoothed Bootstrap.- 3.5.1 Empirical evidences and examples.- 3.5.2 Sample quantiles.- 3.5.3 Remarks.- 3.6 Nonregular Cases.- 3.7 Conclusions and Discussions.- 4. Bootstrap Confidence Sets and Hypothesis Tests.- 4.1 Bootstrap Confidence Sets.- 4.1.1 The bootstrap-t.- 4.1.2 The bootstrap percentile.- 4.1.3 The bootstrap bias-corrected percentile.- 4.1.4 The bootstrap accelerated bias-corrected percentile.- 4.1.5 The hybrid bootstrap.- 4.2 Asymptotic Theory.- 4.2.1 Consistency.- 4.2.2 Accuracy.- 4.2.3 Other asymptotic comparisons.- 4.3 The Iterative Bootstrap and Other Methods.- 4.3.1 The iterative bootstrap.- 4.3.2 Bootstrap calibrating.- 4.3.3 The automatic percentile and variance stabilizing.- 4.3.4 Fixed width bootstrap confidence intervals.- 4.3.5 Likelihood based bootstrap confidence sets.- 4.4 Empirical Comparisons.- 4.4.1 The bootstrap-t, percentile, BC, and BCa.- 4.4.2 The bootstrap and other asymptotic methods.- 4.4.3 The iterative bootstrap and bootstrap calibration.- 4.4.4 Summary.- 4.5 Bootstrap Hypothesis Tests.- 4.5.1 General description.- 4.5.2 Two-sided hypotheses with nuisance parameters.- 4.5.3 Bootstrap distance tests.- 4.5.4 Other results and discussions.- 4.6 Conclusions and Discussions.- 5. Computational Methods.- 5.1 The Delete-1 Jackknife.- 5.1.1 The one-step jackknife.- 5.1.2 Grouping and random subsampling.- 5.2 The Delete-d Jackknife.- 5.2.1 Balanced subsampling.- 5.2.2 Random subsampling.- 5.3 Analytic Approaches for the Bootstrap.- 5.3.1 The delta method.- 5.3.2 Jackknife approximations.- 5.3.3 Saddle point approximations.- 5.3.4 Remarks.- 5.4 Simulation Approaches for the Bootstrap.- 5.4.1 The simple Monte Carlo method.- 5.4.2 Balanced bootstrap resampling.- 5.4.3 Centering after Monte Carlo.- 5.4.4 The linear bootstrap.- 5.4.5 Antithetic bootstrap resampling.- 5.4.6 Importance bootstrap resampling.- 5.4.7 The one-step bootstrap.- 5.5 Conclusions and Discussions.- 6. Applications to Sample Surveys.- 6.1 Sampling Designs and Estimates.- 6.2 Resampling Methods.- 6.2.1 The jackknife.- 6.2.2 The balanced repeated replication.- 6.2.3 Approximated BRR methods.- 6.2.4 The bootstrap.- 6.3 Comparisons by Simulation.- 6.4 Asymptotic Results.- 6.4.1 Assumptions.- 6.4.2 The jackknife and BRR for functions of averages.- 6.4.3 The RGBRR and RSBRR for functions of averages.- 6.4.4 The bootstrap for functions of averages.- 6.4.5 The BRR and bootstrap for sample quantiles.- 6.5 Resampling Under Imputation.- 6.5.1 Hot deck imputation.- 6.5.2 An adjusted jackknife.- 6.5.3 Multiple bootstrap hot deck imputation.- 6.5.4 Bootstrapping under imputation.- 6.6 Conclusions and Discussions.- 7. Applications to Linear Models.- 7.1 Linear Models and Regression Estimates.- 7.2 Variance and Bias Estimation.- 7.2.1 Weighted and unweighted jackknives.- 7.2.2 Three types of bootstraps.- 7.2.3 Robustness and efficiency.- 7.3 Inference and Prediction Using the Bootstrap.- 7.3.1 Confidence sets.- 7.3.2 Simultaneous confidence intervals.- 7.3.3 Hypothesis tests.- 7.3.4 Prediction.- 7.4 Model Selection.- 7.4.1 Cross-validation.- 7.4.2 The bootstrap.- 7.5 Asymptotic Theory.- 7.5.1 Variance estimators.- 7.5.2 Bias estimators.- 7.5.3 Bootstrap distribution estimators.- 7.5.4 Inference and prediction.- 7.5.5 Model selection.- 7.6 Conclusions and Discussions.- 8. Applications to Nonlinear, Nonparametric, and Multivariate Models.- 8.1 Nonlinear Regression.- 8.1.1 Jackknife variance estimators.- 8.1.2 Bootstrap distributions and confidence sets.- 8.1.3 Cross-validation for model selection.- 8.2 Generalized Linear Models.- 8.2.1 Jackknife variance estimators.- 8.2.2 Bootstrap procedures.- 8.2.3 Model selection by bootstrapping.- 8.3 Cox's Regression Models.- 8.3.1 Jackknife variance estimators.- 8.3.2 Bootstrap procedures.- 8.4 Kernel Density Estimation.- 8.4.1 Bandwidth selection by cross-validation.- 8.4.2 Bandwidth selection by bootstrapping.- 8.4.3 Bootstrap confidence sets.- 8.5 Nonparametric Regression.- 8.5.1 Kernel estimates for fixed design.- 8.5.2 Kernel estimates for random regressor.- 8.5.3 Nearest neighbor estimates.- 8.5.4 Smoothing splines.- 8.6 Multivariate Analysis.- 8.6.1 Analysis of covariance matrix.- 8.6.2 Multivariate linear models.- 8.6.3 Discriminant analysis.- 8.6.4 Factor analysis and clustering.- 8.7 Conclusions and Discussions.- 9. Applications to Time Series and Other Dependent Data.- 9.1 m-Dependent Data.- 9.2 Markov Chains.- 9.3 Autoregressive Time Series.- 9.3.1 Bootstrapping residuals.- 9.3.2 Model selection.- 9.4 Other Time Series.- 9.4.1 ARMA(p,q) models.- 9.4.2 Linear regression with time series errors.- 9.4.3 Dynamical linear regression.- 9.5 Stationary Processes.- 9.5.1 Moving block and circular block.- 9.5.2 Consistency of the bootstrap.- 9.5.3 Accuracy of the bootstrap.- 9.5.4 Remarks.- 9.6 Conclusions and Discussions.- 10. Bayesian Bootstrap and Random Weighting.- 10.1 Bayesian Bootstrap.- 10.1.1 Bayesian bootstrap with a noninformative prior.- 10.1.2 Bayesian bootstrap using prior information.- 10.1.3 The weighted likelihood bootstrap.- 10.1.4 Some remarks.- 10.2 Random Weighting.- 10.2.1 Motivation.- 10.2.2 Consistency.- 10.2.3 Asymptotic accuracy.- 10.3 Random Weighting for Functional and Linear Models.- 10.3.1 Statistical functionals.- 10.3.2 Linear models.- 10.4 Empirical Results for Random Weighting.- 10.5 Conclusions and Discussions.- Appendix A. Asymptotic Results.- A.1 Modes of Convergence.- A.2 Convergence of Transformations.- A.4 The Borel-Cantelli Lemma.- A.5 The Law of Large Numbers.- A.6 The Law of the Iterated Logarithm.- A.7 Uniform Integrability.- A.8 The Central Limit Theorem.- A.9 The Berry-Esseen Theorem.- A.10 Edgeworth Expansions.- A.11 Cornish-Fisher Expansions.- Appendix B. Notation.- References.- Author Index.