The impact of speed and bias on the cognitive processes of experts and novices in medical image decision-making

Training individuals to make accurate decisions from medical images is a critical component of education in diagnostic pathology. We describe a joint experimental and computational modeling approach to examine the similarities and differences in the cognitive processes of novice participants and experienced participants (pathology residents and pathology faculty) in cancer cell image identification. For this study we collected a bank of hundreds of digital images that were identified by cell type and classified by difficulty by a panel of expert hematopathologists. The key manipulations in our study included examining the speed-accuracy tradeoff as well as the impact of prior expectations on decisions. In addition, our study examined individual differences in decision-making by comparing task performance to domain general visual ability (as measured using the Novel Object Memory Test (NOMT) (Richler et al. Cognition 166:42–55, 2017). Using signal detection theory and the diffusion decision model (DDM), we found many similarities between experts and novices in our task. While experts tended to have better discriminability, the two groups responded similarly to time pressure (i.e., reduced caution under speed instructions in the DDM) and to the introduction of a probabilistic cue (i.e., increased response bias in the DDM). These results have important implications for training in this area as well as using novice participants in research on medical image perception and decision-making.

[1]  C. Summerfield,et al.  Expectation (and attention) in visual cognition , 2009, Trends in Cognitive Sciences.

[2]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[3]  Ehsan Samei,et al.  The Handbook of Medical Image Perception and Techniques , 2009 .

[4]  Scott D. Brown,et al.  Diffusion Decision Model: Current Issues and History , 2016, Trends in Cognitive Sciences.

[5]  Brandon M. Turner,et al.  Approximate Bayesian computation with differential evolution , 2012 .

[6]  W Kirch,et al.  Misdiagnosis at a university hospital in 4 medical eras. , 1996, Medicine.

[7]  Andrew A Renshaw,et al.  Increasing cytotechnologist workload above 100 slides per day using the ThinPrep imaging system leads to significant reductions in screening accuracy , 2010, Cancer Cytopathology.

[8]  E. Krupinski,et al.  Characterizing the development of visual search expertise in pathology residents viewing whole slide images. , 2013, Human pathology.

[9]  William Hendee,et al.  The Handbook of Medical Image Perception and Techniques. , 2010, Medical physics.

[10]  A V Reed,et al.  Speed-Accuracy Trade-Off in Recognition Memory , 1973, Science.

[11]  R. Ratcliff,et al.  What cognitive processes drive response biases? A diffusion model analysis , 2011, Judgment and Decision Making.

[12]  R. Ratcliff,et al.  The effects of aging on reaction time in a signal detection task. , 2001, Psychology and aging.

[13]  A. Glöckner,et al.  The interplay of experience-based affective and probabilistic cues in decision making: arousal increases when experience and additional cues conflict. , 2011, Experimental psychology.

[14]  Joshua J. Tremel,et al.  Prior probability and feature predictability interactively bias perceptual decisions , 2014, Neuropsychologia.

[15]  Katharina Sonderegger-lseli,et al.  Diagnostic errors in three medical eras: a necropsy study , 2000, The Lancet.

[16]  Kathryn M McDonald,et al.  Changes in rates of autopsy-detected diagnostic errors over time: a systematic review. , 2003, JAMA.

[17]  Elizabeth A Krupinski,et al.  Current perspectives in medical image perception , 2010, Attention, perception & psychophysics.

[18]  Andrew Heathcote,et al.  A new framework for modeling decisions about changing information: The Piecewise Linear Ballistic Accumulator model , 2016, Cognitive Psychology.

[19]  Wayne A. Wickelgren,et al.  Speed-accuracy tradeoff and information processing dynamics , 1977 .

[20]  Andrea Bennett,et al.  ASCP Wage and Vacancy Survey of U.S. Medical Laboratories , 2009 .

[21]  Leslie G. Ungerleider,et al.  The neural systems that mediate human perceptual decision making , 2008, Nature Reviews Neuroscience.

[22]  W. Sommer,et al.  Face and object cognition across adult age. , 2013, Psychology and aging.

[23]  K. Nakayama,et al.  The Cambridge Face Memory Test: Results for neurologically intact individuals and an investigation of its validity using inverted face stimuli and prosopagnosic participants , 2006, Neuropsychologia.

[24]  Solveig Hofvind,et al.  Breast cancer: missed interval and screening-detected cancer at full-field digital mammography and screen-film mammography-- results from a retrospective review. , 2012, Radiology.

[25]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[26]  U. Neisser VISUAL SEARCH. , 1964, Scientific American.

[27]  Kevin Schofield,et al.  Increasing cytotechnologist workload above 100 slides per day using the BD FocalPoint GS imaging system negatively affects screening performance. , 2012, American journal of clinical pathology.

[28]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[29]  J. Bittner,et al.  Cognitive Research : Principles and Implications , 2017 .

[30]  R. Ratcliff,et al.  A Diffusion Model Analysis of the Effects of Aging on Recognition Memory Journal of Memory and Language , 2003 .

[31]  M. Lee,et al.  Bayesian Cognitive Modeling: A Practical Course , 2014 .

[32]  Michael D. Lee,et al.  Bayesian Cognitive Modeling: Parameter estimation , 2014 .

[33]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[34]  E. Krupinski,et al.  Eye-movement study and human performance using telepathology virtual slides: implications for medical education and differences with experience. , 2006, Human pathology.

[35]  L. Goldman,et al.  The value of the autopsy in three medical eras. , 1983, The New England journal of medicine.

[36]  Scott D. Brown,et al.  The Neural Substrate of Prior Information in Perceptual Decision Making: A Model-Based Analysis , 2010, Front. Hum. Neurosci..

[37]  Raymond Bertram,et al.  The Effect of Expertise on Eye Movement Behaviour in Medical Image Perception , 2013, PloS one.

[38]  C. White,et al.  Decomposing bias in different types of simple decisions. , 2014, Journal of experimental psychology. Learning, memory, and cognition.

[39]  C. J. Ravesloot,et al.  How visual search relates to visual diagnostic performance: a narrative systematic review of eye-tracking research in radiology , 2017, Advances in health sciences education : theory and practice.

[40]  J. F. Schouten,et al.  Reaction time and accuracy. , 1967, Acta psychologica.

[41]  Brandon M. Turner,et al.  A generalized, likelihood-free method for posterior estimation , 2014, Psychonomic bulletin & review.

[42]  Anshu,et al.  American Society of Cytopathology , 2002, Acta Cytologica.

[43]  Rankin W. McGugin,et al.  Experience moderates overlap between object and face recognition, suggesting a common ability. , 2014, Journal of vision.

[44]  Michael Hoerger,et al.  When does haste make waste? Speed-accuracy tradeoff, skill level, and the tools of the trade. , 2008, Journal of experimental psychology. Applied.

[45]  N. Hylton MR imaging for the prediction of breast cancer response to neoadjuvant chemotherapy. , 2013, Radiology.

[46]  J COENEGRACHTS,et al.  [Diagnostic errors]. , 1952, Revue medicale de Liege.

[47]  E. Wagenmakers,et al.  Testing theories of post-error slowing , 2011, Attention, perception & psychophysics.

[48]  Isabel Gauthier,et al.  General object recognition is specific: Evidence from novel and familiar objects , 2017, Cognition.

[49]  A. Renshaw,et al.  American society of cytopathology workload recommendations for automated pap test screening: Developed by the productivity and quality assurance in the era of automated screening task force , 2013, Diagnostic cytopathology.

[50]  Clarissa A. Thompson,et al.  Children are not like older adults: a diffusion model analysis of developmental changes in speeded responses. , 2012, Child development.

[51]  Rankin W. McGugin,et al.  High-resolution imaging of expertise reveals reliable object selectivity in the fusiform face area related to perceptual performance , 2012, Proceedings of the National Academy of Sciences.

[52]  Sian L. Beilock,et al.  Haste does not always make waste: Expertise, direction of attention, and speed versus accuracy in performing sensorimotor skills , 2004, Psychonomic bulletin & review.

[53]  Miranda Robertson,et al.  Neural systems , 1977, Nature.

[54]  Rankin W. McGugin,et al.  The Vanderbilt Expertise Test reveals domain-general and domain-specific sex effects in object recognition , 2012, Vision Research.

[55]  Roger Ratcliff,et al.  Individual differences, aging, and IQ in two-choice tasks , 2010, Cognitive Psychology.

[56]  Jim M. Monti,et al.  Expectation and Surprise Determine Neural Population Responses in the Ventral Visual Stream , 2010, The Journal of Neuroscience.

[57]  Jennifer S Trueblood,et al.  Bayesian analysis of the piecewise diffusion decision model , 2018, Behavior research methods.

[58]  Corey J. Bohil,et al.  Base-rate and payoff effects in multidimensional perceptual categorization. , 1998, Journal of Experimental Psychology. Learning, Memory and Cognition.

[59]  William R. Holmes,et al.  A practical guide to the Probability Density Approximation (PDA) with improved implementation and error characterization , 2015 .

[60]  R Ratcliff,et al.  The effects of aging on reaction time in a signal detection task. , 2001, Psychology and aging.

[61]  C. Summerfield,et al.  Expectation in perceptual decision making: neural and computational mechanisms , 2014, Nature Reviews Neuroscience.

[62]  H. C. Sullivan A Changing Workforce Brings New Challenges, Opportunities , 2016 .

[63]  D. Lewin The Future of the Pathology Workforce , 2016 .

[64]  R. Ratcliff,et al.  Bias in the Brain: A Diffusion Model Analysis of Prior Probability and Potential Payoff , 2012, The Journal of Neuroscience.