Lessons on Data Collection and Curation From the NFL Injury Surveillance Program

Background: “Research-ready” evidence platforms that link sports data with anonymized electronic health records (EHRs) or other data are important tools for evaluating injury occurrence in response to changes in games, training, rules, and other factors. While there is agreement that high-quality data are essential, there is little evidence to guide data curation. Hypothesis: We hypothesized that an EHR used in the course of clinical care and curated for research readiness can provide a robust evidence platform. Our purpose was to describe the data curation used for active injury surveillance by the National Football League (NFL). Study Design: Dynamic cohort study. Level of Evidence: Level 2. Methods: Players provide informed consent for research activities through the collective bargaining process. A league-wide EHR is used to record injuries that come to the attention of the teams’ athletic trainers and physicians, NFL medical spotters, or unaffiliated neurotrauma consultants. Information about football activities and injuries are linkable by player, setting, and event to other sports-related data, including game statistics and game-day stadium quality measures, using a unique player identification designed to protect player privacy. Ongoing data curation is used to review data completeness and accuracy and is adjusted over time in response to findings. Results: The core data curation activities include monthly injury summaries to team staff, queries to resolve incomplete reporting, and periodic external checks. Experiences derived from producing more than 100 reports per year on diverse topics are used to update coding training and related guidance documents in response to missing data or inconsistent coding that is observed. Roughly 20% more injuries were recorded for the same “reportable” injuries after switching from targeted reporting to an EHR. Conclusion: Research-ready databases need systematic curation for quality and completeness, along with related action plans. More injuries were reported through EHR than through targeted reporting. Clinical Relevance: Evidence-driven decision-making thrives on reliable data fine-tuned through systematic use, review, and ongoing adjustments to the curation process.

[1]  Robert B. Anderson,et al.  Higher Rates of Lower Extremity Injury on Synthetic Turf Compared With Natural Turf Among National Football League Athletes: Epidemiologic Confirmation of a Biomechanical Hypothesis , 2018, The American journal of sports medicine.

[2]  J. Crandall,et al.  Video Analysis of Reported Concussion Events in the National Football League During the 2015-2016 and 2016-2017 Seasons , 2018, The American journal of sports medicine.

[3]  Nancy A. Dreyer,et al.  Advancing a Framework for Regulatory Use of Real-World Evidence , 2018, Therapeutic innovation & regulatory science.

[4]  Nancy A. Dreyer,et al.  21st Century Patient Registries, Ebook Addendum to Registries for Evaluating Patient Outcomes: A User's Guide, 3rd Edition , 2018 .

[5]  Amy P. Abernethy,et al.  Harnessing the Power of Real‐World Evidence (RWE): A Checklist to Ensure Regulatory‐Grade Data Quality , 2017, Clinical pharmacology and therapeutics.

[6]  R. Califf Benefit-Risk Assessments at the US Food and Drug Administration: Finding the Balance. , 2017, JAMA.

[7]  Petra Kaufmann,et al.  Transforming Evidence Generation to Support Health and Health Care Decisions. , 2016, The New England journal of medicine.

[8]  R. Califf,et al.  Real-World Evidence - What Is It and What Can It Tell Us? , 2016, The New England journal of medicine.

[9]  N. Dreyer,et al.  The fast route to evidence development for value in healthcare , 2016, Current medical research and opinion.

[10]  S. Görtz,et al.  The University of the National Football League: How Technology, Injury Surveillance, and Health Care Have Improved the Safety of America's Game , 2016, The Journal of Knee Surgery.

[11]  Steven R Goldring,et al.  Beyond the decade: strategic priorities to reduce the burden of musculoskeletal disease. , 2013, The Journal of bone and joint surgery. American volume.

[12]  John W Powell,et al.  An Analysis of Specific Lower Extremity Injury Rates on Grass and FieldTurf Playing Surfaces in National Football League Games , 2012, The American journal of sports medicine.

[13]  N. Dreyer,et al.  Registries for Evaluating Patient Outcomes: A User’s Guide , 2010 .

[14]  M Schootman,et al.  A multivariate risk analysis of selected playing surfaces in the National Football League: 1980 to 1989 , 1992, The American journal of sports medicine.

[15]  Duke-Margolis Characterizing RWD Quality and Relevancy for Regulatory Purposes , 2018 .