The bhsdtr package: a general-purpose method of Bayesian inference for signal detection theory models

We describe a novel method of Bayesian inference for hierarchical or non-hierarchical equal variance normal signal detection theory models with one or more criteria. The method is implemented as an open-source R package that uses the state-of-the-art Stan platform for sampling from posterior distributions. Our method can accommodate binary responses as well as additional ratings and an arbitrary number of nested or crossed random grouping factors. The SDT parameters can be regressed on additional predictors within the same model via intermediate unconstrained parameters, and the model can be extended by using automatically generated human-readable Stan code as a template. In the paper, we explain how our method improves on other similar available methods, give an overview of the package, demonstrate its use by providing a real-study data analysis walk-through, and show that the model successfully recovers known parameter values when fitted to simulated data. We also demonstrate that ignoring a hierarchical data structure may lead to severely biased estimates when fitting signal detection theory models.

[1]  Richard D. Morey,et al.  Problematic effects of aggregation in zROC analysis and a hierarchical modeling solution , 2008 .

[2]  P. Mair,et al.  Extended Rasch Modeling: The eRm Package for the Application of IRT Models in R , 2007 .

[3]  Paul-Christian Bürkner,et al.  brms: An R Package for Bayesian Multilevel Models Using Stan , 2017 .

[4]  T. Wickens Elementary Signal Detection Theory , 2001 .

[5]  H. Lau,et al.  A signal detection theoretic approach for estimating metacognitive sensitivity from confidence ratings , 2012, Consciousness and Cognition.

[6]  Gilbert Ritschard,et al.  Coefficient-wise tree-based varying coefficient regression with vcrpart , 2017 .

[7]  M. Overgaard,et al.  Introspection and subliminal perception , 2004 .

[8]  W. W. Peterson,et al.  The theory of signal detectability , 1954, Trans. IRE Prof. Group Inf. Theory.

[9]  D. Luce,et al.  Detection and Recognition " ' , 2006 .

[10]  Don McNicol,et al.  A Primer of Signal Detection Theory , 1976 .

[11]  Lawrence T. DeCarlo,et al.  Signal detection theory and generalized linear models , 1998 .

[12]  S. Sternberg,et al.  Separate modifiability, mental modules, and the use of pure and composite measures to reveal them. , 2001, Acta psychologica.

[13]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[14]  J. Swets,et al.  A decision-making theory of visual detection. , 1954, Psychological review.

[15]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[16]  Paul-Christian Bürkner,et al.  Ordinal Regression Models in Psychology: A Tutorial , 2019, Advances in Methods and Practices in Psychological Science.

[17]  Stephen M Fleming,et al.  HMeta-d: hierarchical Bayesian estimation of metacognitive efficiency from confidence ratings , 2016, bioRxiv.

[18]  Peter D. Kvam,et al.  The Confidence Database , 2020, Nature Human Behaviour.

[19]  J A Swets,et al.  Form of empirical ROCs in discrimination and diagnostic tasks: implications for theory and measurement of performance. , 1986, Psychological bulletin.

[20]  Shravan Vasishth,et al.  Bayesian linear mixed models using Stan: A tutorial for psychologists, linguists, and cognitive scientists , 2015, 1506.06201.

[21]  M. Hautus Calculating estimates of sensitivity from group data: Pooled versus averaged estimators , 1997 .

[22]  N A Macmillan,et al.  Detection theory analysis of group data: estimating sensitivity from average hit and false-alarm rates. , 1985, Psychological bulletin.

[23]  Jun Lu,et al.  An introduction to Bayesian hierarchical models with an application in the theory of signal detection , 2005, Psychonomic bulletin & review.

[24]  John A. Swets,et al.  Evaluation of diagnostic systems : methods from signal detection theory , 1982 .

[25]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[26]  A. Gelman Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[27]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .