The Key Driver Implementation Scale (KDIS) for practice facilitators: Psychometric testing in the “Southeastern collaboration to improve blood pressure control” trial

Background Practice facilitators (PFs) provide tailored support to primary care practices to improve the quality of care delivery. Often used by PFs, the “Key Driver Implementation Scale” (KDIS) measures the degree to which a practice implements quality improvement activities from the Chronic Care Model, but the scale’s psychometric properties have not been investigated. We examined construct validity, reliability, floor and ceiling effects, and a longitudinal trend test of the KDIS items in the Southeastern Collaboration to Improve Blood Pressure Control trial. Methods The KDIS items assess a practice’s progress toward implementing: a clinical information system (using their own data to drive change); standardized care processes; optimized team care; patient self-management support; and leadership support. We assessed construct validity and estimated reliability with a multilevel confirmatory factor analysis (CFA). A trend test examined whether the KDIS items increased over time and estimated the expected number of months needed to move a practice to the highest response options. Results PFs completed monthly KDIS ratings over 12 months for 32 primary care practices, yielding a total of 384 observations. Data was fitted to a unidimensional CFA model; however, parameter fit was modest and could be improved. Reliability was 0.70. Practices started scoring at the highest levels beginning in month 5, indicating low variability. The KDIS items did show an upward trend over 12 months (all p < .001), indicating that practices were increasingly implementing key activities. The expected time to move a practice to the highest response category was 9.1 months for standardized care processes, 10.2 for clinical information system, 12.6 for self-management support, 13.1 for leadership, and 14.3 months for optimized team care. Conclusions The KDIS items showed acceptable reliability, but work is needed in larger sample sizes to determine if two or more groups of implementation activities are being measured rather than one.

[1]  Jacqueline R. Halladay,et al.  Successful Trial of Practice Facilitation for Plan, Do, Study, Act Quality Improvement , 2021, The Journal of the American Board of Family Medicine.

[2]  R. Phillips,et al.  Implementing High-Quality Primary Care , 2021 .

[3]  Theresa L. Walunas,et al.  Identifying Practice Facilitation Delays and Barriers in Primary Care Quality Improvement , 2020, The Journal of the American Board of Family Medicine.

[4]  A. Ehrenberg,et al.  The use of the PARIHS framework in implementation research and practice—a citation analysis of the literature , 2020, Implementation Science.

[5]  Jacqueline R. Halladay,et al.  Practice level factors associated with enhanced engagement with practice facilitators; findings from the heart health now study , 2020, BMC Health Services Research.

[6]  L. McLeod,et al.  State of the psychometric methods: patient-reported outcome measure development and refinement using item response theory , 2019, Journal of patient-reported outcomes.

[7]  A. Leasure,et al.  Use and Effectiveness of the Teach-Back Method in Patient Education and Health Outcomes. , 2019, Federal practitioner : for the health care professionals of the VA, DoD, and PHS.

[8]  Borsika A. Rabin,et al.  RE-AIM Planning and Evaluation Framework: Adapting to New Science and Practice With a 20-Year Review , 2019, Front. Public Health.

[9]  A. Kho,et al.  Impact of Practice Facilitation in Primary Care on Chronic Disease Care Processes and Outcomes: a Systematic Review , 2018, Journal of General Internal Medicine.

[10]  Sera L. Young,et al.  Best Practices for Developing and Validating Scales for Health, Social, and Behavioral Research: A Primer , 2018, Front. Public Health.

[11]  David A. Chambers,et al.  Harnessing the Synergy Between Improvement Science and Implementation Science in Cancer: A Call to Action. , 2018, Journal of oncology practice.

[12]  C. Distefano,et al.  Examination of the Weighted Root Mean Square Residual: Evidence for Trustworthiness? , 2018 .

[13]  Raja A. Cholan,et al.  Practice Facilitator Strategies for Addressing Electronic Health Record Data Challenges for Quality Improvement: EvidenceNOW , 2018, The Journal of the American Board of Family Medicine.

[14]  Sarah J. Shoemaker,et al.  The Capacity of Primary Care for Improving Evidence-Based Care: Early Findings From AHRQ’s EvidenceNOW , 2018, The Annals of Family Medicine.

[15]  B. Weiner,et al.  Organizational Leadership and Adaptive Reserve in Blood Pressure Control: The Heart Health NOW Study , 2018, The Annals of Family Medicine.

[16]  C. Zhan,et al.  EvidenceNOW: Balancing Primary Care Implementation and Implementation Research , 2018, The Annals of Family Medicine.

[17]  Ming-Yuan Chih,et al.  The use of external change agents to promote quality improvement and organizational change in healthcare organizations: a systematic review , 2018, BMC Health Services Research.

[18]  Gillian Harvey,et al.  Enabling Continuous Quality Improvement in Practice: The Role and Contribution of Facilitation , 2017, Front. Public Health.

[19]  Terry K Koo,et al.  A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. , 2016, Journal of chiropractic medicine.

[20]  Gill Harvey,et al.  PARIHS revisited: from heuristic to integrated framework for the successful implementation of knowledge into practice , 2015, Implementation Science.

[21]  Cara C. Lewis,et al.  Outcomes for implementation science: an enhanced systematic review of instruments using evidence-based rating criteria , 2015, Implementation Science.

[22]  Kristopher J Preacher,et al.  Reliability estimation in a multilevel confirmatory factor analysis framework. , 2014, Psychological methods.

[23]  C. Liddy,et al.  Primary care quality improvement from a practice facilitator’s perspective , 2014, BMC Family Practice.

[24]  Mortaza Jamshidian,et al.  MissMech: An R Package for Testing Homoscedasticity, Multivariate Normality, and Missing Completely at Random (MCAR) , 2014 .

[25]  Luci K. Leykum,et al.  A randomized trial of practice facilitation to improve the delivery of chronic illness care in primary care: initial and sustained effects , 2013, Implementation Science.

[26]  Jacqueline R. Halladay,et al.  Facilitators of Transforming Primary Care: A Look Under the Hood at Practice Leadership , 2013, The Annals of Family Medicine.

[27]  N. Baskerville,et al.  Systematic Review and Meta-Analysis of Practice Facilitation Within Primary Care Settings , 2012, The Annals of Family Medicine.

[28]  M. Parchman,et al.  Developing and Running a Primary Care Practice Facilitation Program A HowTo Guide , 2011 .

[29]  Alicia C. Bunger,et al.  Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda , 2010, Administration and Policy in Mental Health and Mental Health Services Research.

[30]  M. Jamshidian,et al.  Tests of Homoscedasticity, Normality, and Missing Completely at Random for Incomplete Multivariate Data , 2010, Psychometrika.

[31]  Richard Scoville,et al.  Designing a large-scale multilevel improvement initiative: the improving performance in practice program. , 2010, The Journal of continuing education in the health professions.

[32]  Jaak Billiet,et al.  A Monte Carlo sample size study: How many countries are needed for accurate multilevel SEM? , 2009 .

[33]  Jo Rycroft-Malone,et al.  Evaluating the successful implementation of evidence into practice using the PARiHS framework: theoretical and practical challenges , 2008, Implementation science : IS.

[34]  R. Grol,et al.  The effects of a team-based continuous quality improvement intervention on the management of primary care: a randomised controlled trial. , 2006, The British journal of general practice : the journal of the Royal College of General Practitioners.

[35]  Rosalie J. Hall,et al.  Applying multilevel confirmatory factor analysis techniques to the study of leadership , 2005 .

[36]  R. Grol,et al.  Improving quality of organizing cardiovascular preventive care in general practice by outreach visitors: a randomized controlled trial. , 2002, Preventive medicine.

[37]  T. Bodenheimer,et al.  Improving primary care for patients with chronic illness. , 2002, JAMA.

[38]  Jo Rycroft-Malone,et al.  Getting evidence into practice: the role and function of facilitation. , 2002, Journal of advanced nursing.

[39]  Brian T. Austin,et al.  Organizing care for patients with chronic illness. , 1996, The Milbank quarterly.

[40]  C. McHorney,et al.  Individual-patient monitoring in clinical practice: are available health status surveys adequate? , 1995, Quality of Life Research.

[41]  Karin E. Johnson,et al.  AHRQ Infrastructure for Maintaining Primary Care Transformation (IMPaCT) Grants , 2015 .

[42]  Jacqueline R. Halladay,et al.  More Extensive Implementation of the Chronic Care Model is Associated with Better Lipid Control in Diabetes , 2014, The Journal of the American Board of Family Medicine.

[43]  Shinyi Wu,et al.  Integrating Chronic Care and Business Strategies in the Safety Net: A Practice Coaching Manual , 2009 .

[44]  Brian T. Austin,et al.  Evidence on the Chronic Care Model in the new millennium. , 2009, Health affairs.

[45]  K. Martinson.  A How-to Guide , 2000 .

[46]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[47]  Michael Eid,et al.  Longitudinal Con rmatory Factor Analysis for Polytomous Item Responses: Model De nition and Model Selection on the Basis of Stochastic Measurement Theory , 1998 .

[48]  T A Louis,et al.  Random effects models with non-parametric priors. , 1992, Statistics in medicine.

[49]  R. Little.  A Test of Missing Completely at Random for Multivariate Data with Missing Values , 1988 .