There has been a surge of availability and use for research of routinely collected electronic health data, such as electronic health records, health administrative data, and disease registries. Symptomatic of this surge, in 2012, Pharmacoepidemiology and Drug Safety (PDS) published a supplemental issue containing several reviews of validated methods for identifying health outcomes using routine health data,1 focusing on databases feeding the US Mini-Sentinel Program.2 In one of the review papers of the PDS Supplement, Carnahan3 acknowledged that while ample validated algorithms exist for major health events, for example, cardiovascular events, validated methods of identifying many health outcomes are lacking. Furthermore, the referenced studies focused on algorithms based on coding sets used in the United States (eg, ICD-9) to identify events from US databases, set within the US health care system. This leaves out an entire segment of routine databases, most notably, Nordic national registries or other European databases such as Clinical Practice Research Datalink (CPRD), The Heatlh Improvement Network (THIN) Hospital Episode Statistics (HES), or PHARMO, all of which are set in health care systems that are differently run and financed than those in the United States. Since other systems function differently, and the databases contain different variables, validation of health status in US data may not always be generalizable.5–9 Many validation studies have been done among these various resources,10–12 but the work is far from complete, as shown in a systematic review of validation studies of the UK-based Clinical Practice Research Datalink, published in 2010.13 Some algorithms may become outdated because of changes in coding or medical practices; new diseases, without clear representation in classification systems, may emerge. Furthermore, in October 2015, the United States adopted ICD-10,14 while ICD-11 is looming on the horizon.15
Clinical Epidemiology has published and continues to publish studies that describe the validity of algorithms in routinely recorded health data, such as validation of medication use in hospitals,16,17 cancer characteristics and complications,18–20 or events related to reproductive and fetal medicine,21,22 to name just a few examples. An “algorithm” in the present context refers to a combination of values of routinely collected variables that allow identification of cases of a given disease or other health event without having to contact or examine the patient. For example, an algorithm based on a combination of diagnostic ICD-10 codes E10-E11 and medication ATC codes A10 may identify patients with diabetes. The commonly evaluated aspects of an algorithm’s validity are positive predictive value (proportion of algorithm-positive patients who truly have the disease of interest) and sensitivity (proportion of patients with the disease of interest who are algorithm-positive), and their counterparts negative predictive value (proportion of algorithm-negative persons without the disease of interest) and specificity (proportion of persons without the disease who are algorithm-negative). Validity of entire data sources is commonly measured by their completeness (proportion of true cases of a disease captured by a data source). A comprehensive review of methods for validating algorithms to identify disease cohorts from health administrative data, with accompanying reporting guidelines for such work, was published by the Journal of Clinical Epidemiology in 2011.23
Clinical Epidemiology is hereby issuing a targeted call for papers that report on results of validation studies. We are interested in publishing both original validation studies and systematic reviews, using various types of reference (“gold”) standards, such as review of medical charts or comparison with other data sources. Several resources are available to guide reporting, including the 2011 guidelines mentioned above,23 as well as the STARD Checklist,24 and the RECORD Checklist.25,26 Please take advantage of these resources in preparing your high-quality submissions.
Some may think of validation work as mundane, a mere poor relative of the “real” original research. We subscribe to a different viewpoint. First, misclassification of study variables threatens the validity of research findings.27 Since epidemiologic research is “an exercise in measurement”,28 high-quality original research is unthinkable without accurate or accurately calibrated instruments. In our editorial experience, evidence of data validity is routinely requested by article referees. Second, following from above, results of validation studies allow epidemiologists to assess the extent of misclassification and estimate its impact on the study results. Third, shining the spotlight on validation studies may activate a feedback loop: physicians may become even more motivated to use systematic coding schemes keeping in mind that the data they feed into the routine databases will be used for research that will ultimately benefit their patients. Last, but not least, validation studies are frequently cited. For example, systematic reviews by Khan et al29 and Herrett et al,13 published in 2010, have already received more than 240 and 350 citations, respectively. We hope you find our arguments compelling and look forward to receiving your validation study submissions.
[1]
Henrik Toft Sørensen,et al.
Validity of the recorded International Classification of Diseases, 10th edition diagnoses codes of bone metastases and skeletal-related events in breast and prostate cancer patients in the Danish National Registry of Patients
,
2009,
Clinical epidemiology.
[2]
Jennifer L Lund,et al.
Validity of the Danish National Registry of Patients for chemotherapy reporting among colorectal cancer patients is high
,
2013,
Clinical epidemiology.
[3]
David Moher,et al.
The REporting of Studies Conducted Using Observational Routinely-Collected Health Data (RECORD) Statement: Methods for Arriving at Consensus and Developing Reporting Guidelines
,
2015,
PloS one.
[4]
Teresa To,et al.
Development and use of reporting guidelines for assessing the quality of validation studies of health administrative data.
,
2011,
Journal of clinical epidemiology.
[5]
M. Schuemie,et al.
Combining electronic healthcare databases in Europe to allow for large‐scale drug safety monitoring: the EU‐ADR Project
,
2011,
Pharmacoepidemiology and drug safety.
[6]
J. Lei,et al.
Combining multiple healthcare databases for postmarketing drug and vaccine safety surveillance: why and how?
,
2014,
Journal of internal medicine.
[7]
Corri Black,et al.
Validity of the General Practice Research Database
,
2003,
Pharmacotherapy.
[8]
J. Ioannidis,et al.
Nationwide Population Science: Lessons From the Taiwan National Health Insurance Research Database.
,
2015,
JAMA internal medicine.
[9]
K Furu,et al.
The Nordic prescription databases as a resource for pharmacoepidemiological research—a literature review
,
2013,
Pharmacoepidemiology and drug safety.
[10]
Mette Søgaard,et al.
Completeness of TNM staging of small-cell and non-small-cell lung cancer in the Danish cancer registry, 2004–2009
,
2012,
Clinical epidemiology.
[11]
R. Carnahan,et al.
Mini‐Sentinel's systematic reviews of validated methods for identifying health outcomes using administrative data: summary of findings and suggestions for future research
,
2012,
Pharmacoepidemiology and drug safety.
[12]
Timothy L Lash,et al.
Validation of spontaneous abortion diagnoses in the Danish National Registry of Patients
,
2010,
Clinical epidemiology.
[13]
L. Smeeth,et al.
Validation and validity of diagnoses in the General Practice Research Database: a systematic review
,
2010,
British journal of clinical pharmacology.
[14]
Henrik Toft Sørensen,et al.
The Danish Civil Registration System as a tool in epidemiology
,
2014,
European Journal of Epidemiology.
[15]
Sandra Kruchov Thygesen,et al.
Positive predictive value of the infant respiratory distress syndrome diagnosis in the Danish National Patient Registry
,
2013,
Clinical epidemiology.
[16]
J. Ludvigsson,et al.
External review and validation of the Swedish national inpatient register
,
2011,
BMC public health.
[17]
Pharmacoepidemiology and drug safety
,
1996,
Pharmacoepidemiology and drug safety.
[18]
Sigrun Alba Johannesdottir Schmidt,et al.
The Danish National Patient Registry: a review of content, data quality, and research potential
,
2015,
Clinical epidemiology.
[19]
P. Rose,et al.
Validity of diagnostic coding within the General Practice Research Database: a systematic review.
,
2010,
The British journal of general practice : the journal of the Royal College of General Practitioners.
[20]
Vera Ehrenstein,et al.
Positive predictive values of the coding for bisphosphonate therapy among cancer patients in the Danish National Patient Registry
,
2012,
Clinical epidemiology.
[21]
T. Stukel,et al.
Importance of accurately identifying disease in studies using electronic health records
,
2010,
BMJ : British Medical Journal.
[22]
S. Greenland,et al.
Causation and causal inference in epidemiology.
,
2005,
American journal of public health.
[23]
Madeleine Svensson,et al.
Evaluation of an algorithm ascertaining cases of osteonecrosis of the jaw in the Swedish National Patient Register
,
2013,
Clinical epidemiology.