Evaluating the effects of cognitive support on psychiatric clinical comprehension

OBJECTIVE Clinicians' attention is a precious resource, which in the current healthcare practice is consumed by the cognitive demands arising from complex patient conditions, information overload, time pressure, and the need to aggregate and synthesize information from disparate sources. The ability to organize information in ways that facilitate the generation of effective diagnostic solutions is a distinguishing characteristic of expert physicians, suggesting that automated systems that organize clinical information in a similar manner may augment physicians' decision-making capabilities. In this paper, we describe the design and evaluation of a theoretically driven cognitive support system (CSS) that assists psychiatrists in their interpretation of clinical cases. The system highlights, and provides the means to navigate to, text that is organized in accordance with a set of diagnostically and therapeutically meaningful higher-level concepts. METHODS AND MATERIALS To evaluate the interface, 16 psychiatry residents interpreted two clinical case scenarios, with and without the CSS. Think-aloud protocols captured during their interpretation of the cases were transcribed and analyzed qualitatively. In addition, the frequency and relative position of content related to key higher-level concepts in a verbal summary of the case were evaluated. In addition the transcripts from both groups were compared to an expert derived reference standard using latent semantic analysis (LSA). RESULTS Qualitative analysis showed that users of the system better attended to specific clinically important aspects of both cases when these were highlighted by the system, and revealed ways in which the system mediates hypotheses generation and evaluation. Analysis of the summary data showed differences in emphasis with and without the system. The LSA analysis suggested users of the system were more "expert-like" in their emphasis, and that cognitive support was more effective in the more complex case. CONCLUSIONS Cognitive support impacts upon clinical comprehension. This appears to be largely helpful, but may also lead to neglect of information (such as the psychosocial history) that the system does not highlight. The results have implications for the design of CSSs for clinical narratives including the role of information organization and textual embellishments for more efficient clinical case presentation and comprehension.

[1]  K. A. Ericsson,et al.  Protocol Analysis: Verbal Reports as Data , 1984 .

[2]  Vimla L. Patel,et al.  The general and specific nature of medical expertise: A critical look. , 1991 .

[3]  Vimla L. Patel,et al.  Identifying reasoning strategies in medical decision making: A methodological guide , 2005, J. Biomed. Informatics.

[4]  L. Brooks,et al.  The development of expertise in dermatology. , 1989, Archives of dermatology.

[5]  M. Chi,et al.  The Nature of Expertise , 1988 .

[6]  Michael W. Berry,et al.  GTP (General Text Parser) Software for Text Mining , 2003 .

[7]  T. Landauer,et al.  A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. , 1997 .

[8]  Vimla L. Patel,et al.  Review: A Primer on Aspects of Cognition for Medical Informatics , 2001, J. Am. Medical Informatics Assoc..

[9]  Vimla L. Patel,et al.  Emerging paradigms of cognition in medical decision-making , 2002, J. Biomed. Informatics.

[10]  Vimla L. Patel,et al.  Customizing clinical narratives for the electronic medical record interface using cognitive methods , 2006, Int. J. Medical Informatics.

[11]  Hans M. Soekkha.  Aviation safety : human factors, system engineering, flight operations, economics, strategies, management , 1997 .

[12]  Vimla L. Patel,et al.  Cognitive Informatics in Health and Biomedicine: Case Studies on Critical Care, Complexity and Errors , 2013 .

[13]  E. Hutchins.  Cognition in the wild , 1995 .

[14]  Peter Gärdenfors,et al.  Conceptual spaces - the geometry of thought , 2000 .

[15]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[16]  V L Patel,et al.  Biomedical knowledge in explanations of clinical problems by medical students , 1988, Medical education.

[17]  Vimla L. Patel,et al.  Simulating expert clinical comprehension: Adapting latent semantic analysis to accurately extract clinical concepts from psychiatric narrative , 2008, J. Biomed. Informatics.

[18]  Trevor Cohen,et al.  Empirical distributional semantics: Methods and biomedical applications , 2009, J. Biomed. Informatics.

[19]  R. Spitzer.  Dsm-IV Casebook: A Learning Companion to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition , 1994 .

[20]  David A. Evans,et al.  Managing coherence and context in medical problem-solving discourse. , 1989 .

[21]  A. D. D. Groot.  Thought and Choice in Chess , 1978 .

[22]  Vimla L. Patel,et al.  Diagnostic Reasoning and Medical Expertise , 1994 .

[23]  A. Treisman,et al.  A feature-integration theory of attention , 1980, Cognitive Psychology.

[24]  W Kintsch,et al.  Understanding and solving word arithmetic problems. , 1985, Psychological review.

[25]  R. Glaser,et al.  Expertise in a complex skill: Diagnosing x-ray pictures. , 1988 .

[26]  Vimla L. Patel,et al.  Considering complexity in healthcare systems , 2011, J. Biomed. Informatics.

[27]  Vimla L. Patel,et al.  Understanding the nature of information seeking behavior in critical care: Implications for the design of health information technology , 2013, Artif. Intell. Medicine.

[28]  Trevor Cohen,et al.  Augmenting expertise: Toward computer-enhanced clinical comprehension , 2007 .

[29]  Vimla L. Patel,et al.  Research Paper: Impact of a Computer-based Patient Record System on Data Collection, Knowledge Organization, and Reasoning , 2000, J. Am. Medical Informatics Assoc..