Selective Correlations - the conditional estimators

The problem of Voodoo correlations is recognized in neuroimaging as the problem of estimating quantities of interest from the same data that was used to select them as interesting. In statistical terminology, the problem of inference following selection from the same data is that of selective inference. Motivated by the unwelcome side-effects of the recommended remedy- splitting the data. A method for constructing confidence intervals based on the correct post-selection distribution of the observations has been suggested recently. We utilize a similar approach in order to provide point estimates that account for a large part of the selection bias. We show via extensive simulations that the proposed estimator has favorable properties, namely, that it is likely to reduce estimation bias and the mean squared error compared to the direct estimator without sacrificing power to detect non-zero correlation as in the case of the data splitting approach. We show that both point estimates and confidence intervals are needed in order to get a full assessment of the uncertainty in the point estimates as both are integrated into the Confidence Calibration Plots proposed recently. The computation of the estimators is implemented in an accompanying software package.

[1]  A. Gelman,et al.  Correlations and Multiple Comparisons in Functional Imaging: A Statistical Perspective (Commentary on Vul et al., 2009) , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[2]  Y. Benjamini,et al.  False Discovery Rate–Adjusted Multiple Confidence Intervals for Selected Parameters , 2005 .

[3]  Thomas E. Nichols,et al.  Commentary on Vul et al.'s (2009) “Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition” , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[4]  Elliot T. Berkman,et al.  Correlations in Social Neuroscience Aren't Voodoo: Commentary on Vul et al. (2009) , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[5]  Brian A. Nosek,et al.  Power failure: why small sample size undermines the reliability of neuroscience , 2013, Nature Reviews Neuroscience.

[6]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[7]  T. Yarkoni Big Correlations in Little Studies: Inflated fMRI Correlations Reflect Low Statistical Power—Commentary on Vul et al. (2009) , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[8]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[9]  Nicole A Lazar,et al.  Discussion of “Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition” by Vul et al. (2009) , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[10]  Thomas E. Nichols,et al.  Everything You Never Wanted to Know about Circular Analysis, but Were Afraid to Ask , 2010, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[11]  Joel B. Greenhouse,et al.  Selection Models and the File Drawer Problem , 1988 .

[12]  K. Fiedler Voodoo Correlations Are Everywhere—Not Only in Neuroscience , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[13]  John D. Storey,et al.  Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach , 2004 .

[14]  H. Pashler,et al.  Puzzlingly High Correlations in fMRI Studies of Emotion, Personality, and Social Cognition 1 , 2009, Perspectives on psychological science : a journal of the Association for Psychological Science.

[15]  Y. Benjamini,et al.  Selection Adjusted Confidence Intervals With More Power to Determine the Sign , 2013 .

[16]  Sabrina M. Tom,et al.  The Neural Basis of Loss Aversion in Decision-Making Under Risk , 2007, Science.

[17]  Yoav Benjamini,et al.  Selective correlations; not voodoo , 2014, NeuroImage.

[18]  J. Mumford,et al.  Independence in ROI analysis: where is the voodoo? , 2009, Social cognitive and affective neuroscience.