Translational Science on Trialome: Retrolective Study of Publishing Results of Clinical Trials

Publication of results of clinical trials, including negative publication, is important to inform clinical practice. Clinical trials registries, such as ClinicalTrials.gov (CT.gov), were created to increase transparency about past and current clinical trials. We linked the CT.gov registry with the PubMed database and investigated the typical time between trial completion and publication of results, and an overall percentage of trials with a linked publication reporting trial results. The mean time to publish was 2.04 years. Of the trials that ultimately publish their results, 80.8% of those do so within 3 years. Only 15.1% of trials published their results with a linked publication within 3 years after completion date, with an additional 3.6% publishing at some later time. Methods: Objective: Identify the percentage of trials in CT.gov that had linked publications in PubMed and estimate the overall publication rate and a 3-year publication rate. Design: We used the entire set of clinical trials from ClinicalTrials.gov and linked them via a trial identifier (NCT ID) with PubMed to retrieve publications reporting results of the analyzed trials. Additional inclusion criteria were completed status and completion date between September 1, 2004 and December 31, 2008. Measurements: We measured time to publish as a time difference between trial completion date (from CT.gov) and publication date (from PubMed).We also looked at proportion of trials that publish within 3 years from trial completion date (and also other intervals ranging from 1 to 7 years after trial completion) Results: We analyzed a set of 14,260 trials that met our inclusion criteria. Of those, 2668 trials published their results by referencing the trial NCT in their publication (PubMed search time was January 4th, 2012). All analyzed trials were allowed at least 3 years after study completion for publication. The mean time to publish was 2.04 years (SD: 1.04). Of the trials that ultimately publish their results, 80.8% of those do so within 3 years. Only 15.2% of trials published their results within 3 years after completion date, with an additional 3.6% publishing at some later time. Our study also generated several important informatics observations relevant to CT.gov data collection, PubMed indexing, keyword assignment and article metadata, as well as some limitations of current data APIs. Discussion: Our study shows that a large proportion of trials do not publish their results with proper reference to the trial identifier. It also reveals informatics challenges in collecting trials metadata (e.g., two recorded trial completion times in CT.gov, capturing all trial references by PubMed from publication abstract (currently only CT.gov and ISRCTN), and CT.gov data export format) and contributes to clinical research informatics efforts to better support the overall research enterprise. Our study has the largest sample size compared with prior studies and it is also the first to report the average time to publish trial results. Our study has several limitations. First, it uses a fully automated method to link trials to publications and it relies on authors to properly link registered trials during their manuscript submission via the NCT trial identifier. In our sample we did not consider trial-publication links submitted to CT.gov by the trial sponsor. One prior study on a much smaller sample (n=677 trials) and with different methodology (2 years allowed for publication) used manual review extended with contacting the PI. They reported that 14.2% of trials had a linked publication with additional 31.8% of trials with publications identified by manual keyword search and by contacting the PI. Second, our study investigated only a single trial registery (CT.gov); however, it is the largest WHO accredited registry with the most advanced data API. Third, we only looked at linked publications indexed in PubMed; however, we believe that researchers searching for trial results are most likely to restrict their search to PubMed only as well. A fourth limitation is that the quality of the NCT ID as a data source has not been previously measured. However, we have studied this in a separate study and found it to be reasonably accurate. Also, based on that study, we chose to use the more precise trial-publication link via NCT ID.