Reliability of SNOMED-CT Coding by Three Physicians using Two Terminology Browsers

SNOMED-CT has been promoted as a reference terminology for electronic health record (EHR) systems. Many important EHR functions are based on the assumption that medical concepts will be coded consistently by different users. This study is designed to measure agreement among three physicians using two SNOMED-CT terminology browsers to encode 242 concepts from five ophthalmology case presentations in a publicly-available clinical journal. Inter-coder reliability, based on exact coding match by each physician, was 44% using one browser and 53% using the other. Intra-coder reliability testing revealed that a different SNOMED-CT code was obtained up to 55% of the time when the two browsers were used by one user to encode the same concept. These results suggest that the reliability of SNOMED-CT coding is imperfect, and may be a function of browsing methodology. A combination of physician training, terminology refinement, and browser improvement may help increase the reproducibility of SNOMED-CT coding.

[1]  L. Sharp,et al.  Expert agreement in Current Procedural Terminology evaluation and management coding. , 2002, Archives of internal medicine.

[2]  Daniel Lorence,et al.  Regional Variation in Medical Classification Agreement: Benchmarking the Coding Gap , 2003, Journal of Medical Systems.

[3]  Justin Starren,et al.  Representation of ophthalmology concepts by electronic systems: adequacy of controlled medical terminologies. , 2005, Ophthalmology.

[4]  Justin Starren,et al.  Representation of ophthalmology concepts by electronic systems: intercoder agreement among physicians using controlled terminologies. , 2006, Ophthalmology.

[5]  P. Dodek,et al.  Discordance between physicians and coders in assignment of diagnoses. , 1999, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[6]  Kent A. Spackman,et al.  Review: Representing Thoughts, Words, and Things in the UMLS , 1998, J. Am. Medical Informatics Assoc..

[7]  A. Grove,et al.  Efficiency in the health care industries: a view from the outside. , 2005, JAMA.

[8]  P Carpenter,et al.  Phase II evaluation of clinical coding schemes: completeness, taxonomy, mapping, definitions, and clarity. CPRI Work Group on Codes and Structures. , 1997, Journal of the American Medical Informatics Association : JAMIA.

[9]  Susan A. Caldwell,et al.  The completeness of existing lexicons for representing radiology report information. , 2002, Journal of digital imaging.