Bayesian Inference for Correlations in the Presence of Measurement Error and Estimation Uncertainty

Whenever parameter estimates are uncertain or observations are contaminated by measurement error, the Pearson correlation coefficient can severely underestimate the true strength of an association. Various approaches exist for inferring the correlation in the presence of estimation uncertainty and measurement error, but none are routinely applied in psychological research. Here we focus on a Bayesian hierarchical model proposed by Behseta, Berdyyeva, Olson, and Kass (2009) that allows researchers to infer the underlying correlation between error-contaminated observations. We show that this approach may be also applied to obtain the underlying correlation between uncertain parameter estimates as well as the correlation between uncertain parameter estimates and noisy observations. We illustrate the Bayesian modeling of correlations with two empirical data sets; in each data set, we first infer the posterior distribution of the underlying correlation and then compute Bayes factors to quantify the evidence that the data provide for the presence of an association.

[1]  D. Levine Introduction to Neural and Cognitive Modeling , 2018 .

[2]  Alexander Etz,et al.  Introduction to Bayesian Inference for Psychology , 2018, Psychonomic bulletin & review.

[3]  Brandon M. Turner,et al.  Factor analysis linking functions for simultaneously modeling neural and behavioral data , 2017, NeuroImage.

[4]  Eric-Jan Wagenmakers,et al.  A Tutorial on Fisher Information , 2017, 1705.01064.

[5]  S. Lilienfeld,et al.  Psychological Science Under Scrutiny: Recent Challenges and Proposed Solutions , 2017 .

[6]  M. Lee,et al.  Determining informative priors for cognitive models , 2017, Psychonomic Bulletin & Review.

[7]  Eric-Jan Wagenmakers,et al.  An evaluation of alternative methods for testing hypotheses, from the perspective of Harold Jeffreys , 2016 .

[8]  E. Wagenmakers,et al.  Harold Jeffreys’s default Bayes factor hypothesis tests: Explanation, extension, and application in psychology , 2016 .

[9]  Tal Yarkoni,et al.  Statistically Controlling for Confounding Constructs Is Harder than You Think , 2016, PloS one.

[10]  Eric-Jan Wagenmakers,et al.  Analytic posteriors for Pearson's correlation coefficient , 2015, Statistica Neerlandica.

[11]  C. Spearman The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[12]  L. Held,et al.  Bayesian analysis of measurement error models using integrated nested Laplace approximations , 2015 .

[13]  Richard D. Morey,et al.  Simple relation between Bayesian order-restricted and point-null hypothesis tests , 2014 .

[14]  Joachim Vandekerckhove,et al.  A cognitive latent variable model for the simultaneous analysis of behavioral and personality data. , 2014 .

[15]  Kristopher J Preacher,et al.  Manifest variable path analysis: potentially serious and misleading consequences due to uncorrected measurement error. , 2014, Psychological methods.

[16]  R. Ratcliff,et al.  Modeling simple driving tasks with a one-boundary diffusion model , 2013, Psychonomic Bulletin & Review.

[17]  E. Wagenmakers,et al.  Bayesian Estimation of Multinomial Processing Tree Models with Heterogeneity in Participants and Items , 2013, Psychometrika.

[18]  David J. Lunn,et al.  The BUGS Book: A Practical Introduction to Bayesian Analysis , 2013 .

[19]  Joachim Vandekerckhove,et al.  Extending JAGS: A tutorial on adding custom distributions to JAGS (with a diffusion model example) , 2013, Behavior Research Methods.

[20]  M. Lee,et al.  Using priors to formalize theory: Optimal attention and the generalized context model , 2012, Psychonomic Bulletin & Review.

[21]  Sik-Yum Lee,et al.  A tutorial on the Bayesian approach for analyzing structural equation models , 2012 .

[22]  Andreas Glöckner,et al.  Cognitive models of risky choice: Parameter stability and predictive accuracy of prospect theory , 2012, Cognition.

[23]  Don van Ravenzwaaij,et al.  An integrated perspective on the relation between response speed and intelligence , 2011, Cognition.

[24]  Z. Dienes Bayesian Versus Orthodox Statistics: Which Side Are You On? , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[25]  M. Lee How cognitive modeling can benefit from hierarchical Bayesian models. , 2011 .

[26]  Simon Farrell,et al.  Computational Modeling in Cognition: Principles and Practice , 2010 .

[27]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[28]  Eric-Jan Wagenmakers,et al.  An encompassing prior generalization of the Savage-Dickey density ratio , 2010, Comput. Stat. Data Anal..

[29]  E. Wagenmakers,et al.  Bayesian hypothesis testing for psychologists: A tutorial on the Savage–Dickey method , 2010, Cognitive Psychology.

[30]  Roger Ratcliff,et al.  Individual differences, aging, and IQ in two-choice tasks , 2010, Cognitive Psychology.

[31]  John P. Buonaccorsi,et al.  Measurement Error: Models, Methods, and Applications , 2010 .

[32]  Andrew Thomas,et al.  The BUGS project: Evolution, critique and future directions , 2009, Statistics in medicine.

[33]  E. Wagenmakers,et al.  Psychological interpretation of the ex-Gaussian and shifted Wald parameters: A diffusion model analysis , 2009, Psychonomic bulletin & review.

[34]  Eric-Jan Wagenmakers,et al.  Methodological and empirical developments for the Ratcliff diffusion model of response times and accuracy , 2009 .

[35]  J. Hilbe Data Analysis Using Regression and Multilevel/Hierarchical Models , 2009 .

[36]  R. Kass,et al.  Bayesian correction for attenuation of correlation in multi-trial spike count data. , 2009, Journal of neurophysiology.

[37]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[38]  David B Dunson,et al.  Default Prior Distributions and Efficient Posterior Computation in Bayesian Factor Analysis , 2009, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[39]  Casimir J. H. Ludwig,et al.  Bayesian and maximum likelihood estimation of hierarchical response time models , 2008, Psychonomic bulletin & review.

[40]  M. Aitkin,et al.  Bayes factors: Prior sensitivity and model generalizability , 2008 .

[41]  Jie W Weiss,et al.  Bayesian Statistical Inference for Psychological Research , 2008 .

[42]  Snigdhansu Chatterjee,et al.  Structural Equation Modeling, A Bayesian Approach , 2008, Technometrics.

[43]  Jeffrey N. Rouder,et al.  A hierarchical process-dissociation model. , 2008, Journal of experimental psychology. General.

[44]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[45]  Klaus Oberauer,et al.  Individual differences in components of reaction time distributions and their relations to working memory and intelligence. , 2007, Journal of experimental psychology. General.

[46]  Jun Lu,et al.  Signal Detection Models with Random Participant and Item Effects , 2007 .

[47]  Verena D. Schmittmann,et al.  The 20-Minute Version as a Predictor of the Raven Advanced Progressive Matrices Test , 2006 .

[48]  K. C. Klauer Hierarchical Multinomial Processing Tree Models: A Latent-Trait Approach , 2006 .

[49]  Jun Lu,et al.  An introduction to Bayesian hierarchical models with an application in the theory of signal detection , 2005, Psychonomic bulletin & review.

[50]  Eric P. Charles,et al.  The correction for attenuation due to measurement error: clarifying concepts and creating confidence sets. , 2005, Psychological methods.

[51]  Jeffrey N. Rouder,et al.  A hierarchical model for estimating response time distributions , 2005, Psychonomic bulletin & review.

[52]  Andrew Gelman,et al.  R2WinBUGS: A Package for Running WinBUGS from R , 2005 .

[53]  A. Voss,et al.  Interpreting the parameters of the diffusion model: An empirical validation , 2004, Memory & cognition.

[54]  P. Levy Measurement Error and Misclassification in Statistics and Epidemiology: Impacts and Bayesian Adjustments , 2004 .

[55]  Jeffrey N. Rouder,et al.  A hierarchical bayesian statistical framework for response time distributions , 2003 .

[56]  R. Ratcliff,et al.  A diffusion model analysis of the effects of aging on letter discrimination. , 2003, Psychology and aging.

[57]  I. J. Myung,et al.  Tutorial on maximum likelihood estimation , 2003 .

[58]  Stephen J. Ganocy,et al.  Bayesian Statistical Modelling , 2002, Technometrics.

[59]  H. Luczak Cognitive Modeling , 2000 .

[60]  Sudhir Gupta,et al.  Statistical Regression With Measurement Error , 1999, Technometrics.

[61]  J. Hintze,et al.  Violin plots : A box plot-density trace synergism , 1998 .

[62]  Dani Gamerman,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 1997 .

[63]  S Richardson,et al.  A Bayesian approach to measurement error problems in epidemiology using conditional independence models. , 1993, American journal of epidemiology.

[64]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[65]  A. Tversky,et al.  Advances in prospect theory: Cumulative representation of uncertainty , 1992 .

[66]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[67]  T Nettelbeck,et al.  Perceptual Indices of Performance: The Measurement of ‘Inspection Time’ and ‘Noise’ in the Visual System , 1972, Perception.

[68]  J. Dickey,et al.  The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain , 1970 .

[69]  R. Hohle INFERRED COMPONENTS OF REACTION TIMES AS FUNCTIONS OF FOREPERIOD DURATION. , 1965, Journal of experimental psychology.

[70]  M. Bartlett A comment on D. V. Lindley's statistical paradox , 1957 .

[71]  E. Wagenmakers,et al.  UvA-DARE ( Digital Academic Repository ) Using Bayesian regression to test hypotheses about relationships between parameters and covariates in cognitive models , 2018 .

[72]  Brandon M. Turner,et al.  A flexible and efficient hierarchical Bayesian approach to the exploration of individual differences in cognitive-model-based neuroscience. , 2018 .

[73]  Jeffrey N. Rouder,et al.  The need for Bayesian hypothesis testing in psychological science , 2017 .

[74]  Andrea Faber,et al.  Statistical Regression With Measurement Error , 2016 .

[75]  Arthur R. Jensen The 9 Factor: The Science of Mental Ability , 2015 .

[76]  E. Wagenmakers,et al.  An Introduction to Model-Based Cognitive Neuroscience , 2015, Springer New York.

[77]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[78]  Rick H. Hoyle,et al.  Handbook of structural equation modeling , 2012 .

[79]  Alexander Kukush,et al.  Measurement Error Models , 2011, International Encyclopedia of Statistical Science.

[80]  Roger Ratcliff,et al.  A diffusion model explanation of the worst performance rule for reaction time and IQ. , 2008, Intelligence.

[81]  David B. Dunson,et al.  Bayesian Structural Equation Modeling , 2007 .

[82]  Michael A. West,et al.  BAYESIAN MODEL ASSESSMENT IN FACTOR ANALYSIS , 2004 .

[83]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[84]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[85]  Xiao-Li Meng,et al.  POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES , 1996 .

[86]  A. Raftery,et al.  Bayes factors , 1995 .

[87]  J. Raven,et al.  Manual for Raven's progressive matrices and vocabulary scales , 1962 .

[88]  L. L. Thurstone,et al.  The correction for attenuation. , 1931 .

[89]  A. N. Kolmogorov,et al.  Theory of Probability , 1929, Nature.