Assessing the readability of ClinicalTrials.gov

OBJECTIVE ClinicalTrials.gov serves critical functions of disseminating trial information to the public and helping the trials recruit participants. This study assessed the readability of trial descriptions at ClinicalTrials.gov using multiple quantitative measures. MATERIALS AND METHODS The analysis included all 165,988 trials registered at ClinicalTrials.gov as of April 30, 2014. To obtain benchmarks, the authors also analyzed 2 other medical corpora: (1) all 955 Health Topics articles from MedlinePlus and (2) a random sample of 100,000 clinician notes retrieved from an electronic health records system intended for conveying internal communication among medical professionals. The authors characterized each of the corpora using 4 surface metrics, and then applied 5 different scoring algorithms to assess their readability. The authors hypothesized that clinician notes would be most difficult to read, followed by trial descriptions and MedlinePlus Health Topics articles. RESULTS Trial descriptions have the longest average sentence length (26.1 words) across all corpora; 65% of their words used are not covered by a basic medical English dictionary. In comparison, average sentence length of MedlinePlus Health Topics articles is 61% shorter, vocabulary size is 95% smaller, and dictionary coverage is 46% higher. All 5 scoring algorithms consistently rated CliniclTrials.gov trial descriptions the most difficult corpus to read, even harder than clinician notes. On average, it requires 18 years of education to properly understand these trial descriptions according to the results generated by the readability assessment algorithms. DISCUSSION AND CONCLUSION Trial descriptions at CliniclTrials.gov are extremely difficult to read. Significant work is warranted to improve their readability in order to achieve CliniclTrials.gov's goal of facilitating information dissemination and subject recruitment.

[1]  Mark Garner,et al.  A framework for the evaluation of patient information leaflets , 2012, Health expectations : an international journal of public participation in health care and health policy.

[2]  Benard P Dreyer,et al.  Readability, Suitability, and Characteristics of Asthma Action Plans: Examination of Factors That May Impair Understanding , 2013, Pediatrics.

[3]  E. J. Mayeaux,et al.  Reading ability of parents compared with reading level of pediatric patient education materials. , 1994, Pediatrics.

[4]  Graciela Rosemblat,et al.  Text Features and Readability: Expert Evaluation of Consumer Health Text , 2006 .

[5]  Alla Keselman,et al.  A classification of errors in lay comprehension of medical documents , 2012, J. Biomed. Informatics.

[6]  Michael Mackert,et al.  Readability, suitability, and health content assessment of web-based patient education materials on colorectal cancer screening. , 2014, Gastrointestinal endoscopy.

[7]  G. Harry McLaughlin,et al.  SMOG Grading - A New Readability Formula. , 1969 .

[8]  P. A. Kelly,et al.  Readability of pediatric biomedical research informed consent forms. , 1990, Pediatrics.

[9]  J. Chall,et al.  A FORMULA FOR PREDICTING READABILITY , 1948 .

[10]  Alla Keselman,et al.  Beyond Readability: Investigating Coherence of Clinical Text for Consumers , 2011, Journal of medical Internet research.

[11]  Kai Zheng,et al.  Applying Multiple Methods to Assess the Readability of a Large Corpus of Medical Documents , 2013, MedInfo.

[12]  S. Piantadosi,et al.  Are informed consent forms that describe clinical oncology research protocols readable by most patients and their families? , 1994, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[13]  John Hoey,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors. , 2004, JAMA.

[14]  Philip J. Gregory,et al.  Readability of Consumer Health Information on the Internet: A Comparison of U.S. Government–Funded and Commercially Funded Websites , 2012, Journal of health communication.

[15]  Michael K Paasche-Orlow,et al.  Readability standards for informed-consent forms as compared with actual readability. , 2003, The New England journal of medicine.

[16]  K D Hopper,et al.  Informed consent forms for clinical and research imaging procedures: how much do patients understand? , 1995, AJR. American journal of roentgenology.

[17]  Kai Zheng,et al.  Bootstrapping a de-identification system for narrative patient records: Cost-performance tradeoffs , 2013, Int. J. Medical Informatics.

[18]  Jim Warren,et al.  Assessing text characteristics of electronic discharge summaries and their implications for patient readability , 2010 .

[19]  Lei Yang,et al.  Voice-dictated versus typed-in clinician notes: linguistic properties and the potential implications on natural language processing. , 2011, AMIA ... Annual Symposium proceedings. AMIA Symposium.

[20]  Ken Resnicow,et al.  Racial/ethnic differences in clinical trial enrollment, refusal rates, ineligibility, and reasons for decline among patients at sites in the National Cancer Institute's Community Cancer Centers Program , 2014, Cancer.

[21]  Douglas G Altman,et al.  Systematic reviews in health care: Assessing the quality of controlled clinical trials. , 2001, BMJ.

[22]  David R. Hansberry,et al.  A comparative analysis of the quality of patient education materials from medical specialties. , 2013, JAMA internal medicine.

[23]  R. P. Fishburne,et al.  Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel , 1975 .

[24]  Alla Keselman,et al.  Text Characteristics of Clinical Reports and Their Implications for the Readability of Personal Health Records , 2007, MedInfo.

[25]  T M Grundner,et al.  On the readability of surgical consent forms. , 1980, The New England journal of medicine.

[26]  Meade Cd,et al.  Consent forms: how to determine and improve their readability. , 1992 .

[27]  H. Taub,et al.  Readability of informed consent forms for research in a Veterans Administration medical center. , 1983, JAMA.

[28]  Alla Keselman,et al.  Beyond Surface Characteristics: A New Health Text-Specific Readability Measurement , 2007, AMIA.

[29]  Marli Terblanche,et al.  Examining the readability of patient-informed consent forms , 2010 .

[30]  P. Ley,et al.  The use of readability formulas in health care , 1996 .

[31]  R. Gunning.  The Technique of Clear Writing. , 1968 .

[32]  Lynette Hirschman,et al.  The MITRE Identification Scrubber Toolkit: Design, training, and assessment , 2010, Int. J. Medical Informatics.

[33]  John Hoey,et al.  Clinical trial registration: a statement from the International Committee of Medical Journal Editors. , 2004, JAMA.

[34]  D M D'Alessandro,et al.  The readability of pediatric patient education materials on the World Wide Web. , 2001, Archives of pediatrics & adolescent medicine.