Measuring the Delivery of Complex Interventions through Electronic Medical Records: Challenges and Lessons Learned

Background: Health services and implementation researchers often seek to capture the implementation process of complex interventions yet explicit guidance on how to capture this process is limited. Medical record review is a commonly used methodology, especially when used as a proxy for provider behavior, with recognized benefits and limitations. The purpose of this study was to test the feasibility of chart review to measure implementation and offer recommendations for future researchers using this method to capture the implementation process. Methods: Grounded in qualitative research methods, we measured the implementation of a transitional care intervention for older adults with dementia being discharged from the hospital. We adapted the operationalization of the intervention’s components to suit chart review methods, sought input from hospital providers before and after data collection, and assessed the agreement between the results of our chart review and provider-report. Findings: We believe chart review can be used effectively as a method for capturing the implementation process and provide future researchers with a list of recommendations based on our experience including understanding the nuance between data extraction versus data abstraction, allowing for large amounts of data not pre-specified in the data collection instrument to be collected, and purposefully and iteratively engaging the providers who are entering data into the chart. Major Themes: Measuring the implementation of complex interventions is a cornerstone in health services research and with the relative convenience and low costs of using chart data, we believe with more use and refinement this methodology could emerge as a valuable and widely used method in the field.