A New u‐Statistic with Superior Design Sensitivity in Matched Observational Studies

In an observational or nonrandomized study of treatment effects, a sensitivity analysis indicates the magnitude of bias from unmeasured covariates that would need to be present to alter the conclusions of a naïve analysis that presumes adjustments for observed covariates suffice to remove all bias. The power of sensitivity analysis is the probability that it will reject a false hypothesis about treatment effects allowing for a departure from random assignment of a specified magnitude; in particular, if this specified magnitude is "no departure" then this is the same as the power of a randomization test in a randomized experiment. A new family of u-statistics is proposed that includes Wilcoxon's signed rank statistic but also includes other statistics with substantially higher power when a sensitivity analysis is performed in an observational study. Wilcoxon's statistic has high power to detect small effects in large randomized experiments-that is, it often has good Pitman efficiency-but small effects are invariably sensitive to small unobserved biases. Members of this family of u-statistics that emphasize medium to large effects can have substantially higher power in a sensitivity analysis. For example, in one situation with 250 pair differences that are Normal with expectation 1/2 and variance 1, the power of a sensitivity analysis that uses Wilcoxon's statistic is 0.08 while the power of another member of the family of u-statistics is 0.66. The topic is examined by performing a sensitivity analysis in three observational studies, using an asymptotic measure called the design sensitivity, and by simulating power in finite samples. The three examples are drawn from epidemiology, clinical medicine, and genetic toxicology.

[1]  R. Kronmal,et al.  Assessing the sensitivity of regression results to unmeasured confounders in observational studies. , 1998, Biometrics.

[2]  T. Speed,et al.  On the Application of Probability Theory to Agricultural Experiments. Essay on Principles. Section 9 , 1990 .

[3]  W L WATSON,et al.  Smoking and lung cancer , 1954, Cancer.

[4]  Paul R Rosenbaum,et al.  Confidence Intervals for Uncommon but Dramatic Responses to Treatment , 2007, Biometrics.

[5]  S. Marcus,et al.  Using Omitted Variable Bias to Assess Uncertainty in the Estimation of an AIDS Education Treatment Effect , 1997 .

[6]  W. L. Owens,et al.  Lead absorption in children of employees in a lead-related industry. , 1982, American journal of epidemiology.

[7]  Takashi Yanagawa,et al.  Case-control studies: Assessing the effect of a confounding factor , 1984 .

[8]  P. Rosenbaum,et al.  Minimum Distance Matched Sampling With Fine Balance in an Observational Study of Treatment for Ovarian Cancer , 2007 .

[9]  Dylan S. Small,et al.  Split Samples and Design Sensitivity in Observational Studies , 2009 .

[10]  E. Stuart,et al.  Impacts of age of onset of substance use disorders on risk of adult incarceration among disadvantaged urban youth: a propensity score matching approach. , 2008, Drug and alcohol dependence.

[11]  P. Rosenbaum,et al.  Amplification of Sensitivity Analysis in Matched Observational Studies , 2009, Journal of the American Statistical Association.

[12]  J. Robins,et al.  Sensitivity Analysis for Selection bias and unmeasured Confounding in missing Data and Causal inference models , 2000 .

[13]  S. Greenhouse,et al.  Jerome Cornfield's contributions to epidemiology. , 1982, Biometrics.

[14]  Paul R. Rosenbaum,et al.  Design Sensitivity and Efficiency in Observational Studies , 2010 .

[15]  J. Copas,et al.  Local sensitivity approximations for selectivity bias , 2001 .

[16]  Dylan S. Small,et al.  Sensitivity Analysis for Instrumental Variables Regression With Overidentifying Restrictions , 2007 .

[17]  K E Warner,et al.  Smoking and lung cancer: an overview. , 1984, Cancer research.

[18]  T. DiPrete,et al.  7. Assessing Bias in the Estimation of Causal Effects: Rosenbaum Bounds on Matching Estimators and Instrumental Variables Estimation with Imperfect Instruments , 2004 .

[19]  E. C. Hammond,et al.  Smoking and lung cancer: recent evidence and a discussion of some questions. , 1959, Journal of the National Cancer Institute.

[20]  W. Stephenson,et al.  A General Class of One-Sample Nonparametric Test Statistics Based on Subsamples , 1981 .

[21]  P. Rosenbaum.  Design of Observational Studies , 2009, Springer Series in Statistics.

[22]  G. W. Imbens.  Sensitivity to Exogeneity Assumptions in Program Evaluation , 2003 .

[23]  C. Geiss,et al.  An introduction to probability theory , 2008 .

[24]  W. Hoeffding.  A Class of Statistics with Asymptotically Normal Distribution , 1948 .

[25]  P. Rosenbaum.  Covariance Adjustment in Randomized Experiments and Observational Studies , 2002 .

[26]  Sebastian Schneeweiss,et al.  Selective prescribing led to overestimation of the benefits of lipid-lowering drugs. , 2006, Journal of clinical epidemiology.

[27]  J. I.  The Design of Experiments , 1936, Nature.

[28]  W. R. van Zwet,et al.  Asymptotic Expansions for the Power of Distributionfree Tests in the Two-Sample Problem , 1976 .

[29]  P. Sen,et al.  Theory of rank tests , 1969 .

[30]  Binbing Yu,et al.  Sensitivity analysis for trend tests: application to the risk of radiation exposure. , 2005, Biostatistics.

[31]  D. Rubin.  Estimating causal effects of treatments in randomized and nonrandomized studies. , 1974 .

[32]  Paul R. Rosenbaum,et al.  Design sensitivity in observational studies , 2004 .

[33]  W. Conover,et al.  Locally most powerful tests for detecting treatment effects when only a subset of patients can be expected to "respond" to treatment. , 1988, Biometrics.

[34]  Thomas P. Hettmansperger,et al.  Statistical inference based on ranks , 1985 .

[35]  J. Thigpen.  Does Ovarian Cancer Treatment and Survival Differ by the Specialty Providing Chemotherapy , 2008 .

[36]  Jean D. Gibbons,et al.  Concepts of Nonparametric Theory , 1981 .