Validity and usability of a professional association’s web-based knowledge translation portal: American Physical Therapy Association’s PTNow.org

BackgroundPTNow.org is an evidence-based, on-line portal created by a professional membership association to promote use of evidence in practice and to help decrease unwarranted variation in practice. The site contains synthesis documents designed to promote efficient clinical reasoning. These documents were written and peer-reviewed by teams of content experts and master clinicians. The purpose of this paper is to report on the content and construct validity as well as usability of the site.MethodsPhysical therapist participants used clinical summaries (available in 3 formats--as a full summary with hyperlinks, “quick takes” with hyperlinks, and a portable two-page version) on the PTNow.org site to answer knowledge acquisition and clinical reasoning questions related to four patient scenarios. They also responded to questions about ease of use related to website navigation and about format and completeness of information using a 1–5 Likert scale. Responses were coded to reflect how participants used the site and then were summarized descriptively. Preferences for clinical summary format were analyzed using an analysis of variance (ANOVA) and a Dunnett T3 post hoc analysis.ResultsSeventeen participants completed the study. Clinical relevance and completeness ratings by experienced clinicians, which were used as the measure of content validity, ranged from 3.1 to 4.6 on a 5 point scale. Construct validity based on the information on the PTNow.org site was supported for knowledge acquisition questions 66 % of the time and for clinical reasoning questions 40 % of the time. Usability ratings for the full clinical summary were 4.6 (1.2); for the quick takes, 3.5 (.98); and for the portable clinical summary, 4.0 (.45). Participants preferred the full clinical summary over the other two formats (F = 5.908, P = 0.007). One hundred percent of the participants stated that they would recommend the PTNow site to their colleagues.ConclusionPrelimary evidence supported both content validity and construct validity of knowledge acquisition, and partially supported construct validity of clinical reasoning for the clinical summaries on the PTNow.org site. Usability was supported, with users preferring the full clinical summary over the other two formats. Iterative design is ongoing.

[1]  Debora Shaw,et al.  Handbook of usability testing: How to plan, design, and conduct effective tests , 1996 .

[2]  David L. Sackett,et al.  Evidence based medicine: What it is and what it isn't (reprinted from BMJ, vol 312, pg 71-72, 1996) , 2007 .

[3]  S. Straus,et al.  Lost in knowledge translation: Time for a map? , 2006, The Journal of continuing education in the health professions.

[4]  D. Cook,et al.  Internet-based learning in the health professions: a meta-analysis. , 2008, JAMA.

[5]  T. Hoffmann,et al.  Australian occupational therapists' use of an online evidence-based practice database (OTseeker). , 2005, Health information and libraries journal.

[6]  S. Straus,et al.  Strategies for rehabilitation professionals to move evidence-based knowledge into practice: a systematic review. , 2009, Journal of rehabilitation medicine.

[7]  Megan Smith,et al.  Effect of experience on clinical decision making by cardiorespiratory physiotherapists in acute care settings , 2010, Physiotherapy theory and practice.

[8]  J. Grimshaw,et al.  Knowledge translation of research findings , 2012, Implementation Science.

[9]  J. Grimshaw,et al.  Getting evidence into practice: the work of the Cochrane Effective Practice and Organization of care Group (EPOC). , 2001, The Journal of continuing education in the health professions.

[10]  S. Fruth,et al.  The influence of a topic-specific, research-based presentation on physical therapists' beliefs and practices regarding evidence-based practice , 2010, Physiotherapy theory and practice.

[11]  Leslie G. Portney Dpt PhD Fapta,et al.  Foundations of Clinical Research: Applications to Practice , 2015 .

[12]  Rand J. Spiro,et al.  Cognitive flexibility theory : advanced knowledge acquisition in ill-structured domains , 1988 .

[13]  D. Jette,et al.  Evidence-based practice: beliefs, attitudes, knowledge, and behaviors of physical therapists. , 2003, Physical therapy.

[14]  Farah Magrabi,et al.  Clinicians' Assessments of the Usefulness of Online Evidence to Answer Clinical Questions , 2004, MedInfo.

[15]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[16]  S. Jaglal,et al.  Practitioner and Organizational Barriers to Evidence-based Practice of Physical Therapists for People With Stroke , 2007, Physical Therapy.

[17]  Nancy Santesso,et al.  Knowledge for knowledge translation: The role of the Cochrane Collaboration , 2006, The Journal of continuing education in the health professions.

[18]  L. Ward,et al.  Knowledge in the Palm of your hands: PDAs in the clinical setting. , 2006, Health information and libraries journal.

[19]  J. M. Christian Bastien,et al.  Usability testing: a review of some methodological and technical aspects of the method , 2010, Int. J. Medical Informatics.

[20]  Mark Chignell,et al.  Usability testing of two e-learning resources: methods to maximize potential for clinician use. , 2012, Journal of rehabilitation medicine.

[21]  A. Heinemann,et al.  Development and use of a knowledge translation tool: the rehabilitation measures database. , 2014, Archives of physical medicine and rehabilitation.

[22]  Robin D. Washington.  Review of: Foundations of Clinical Research Applications to Practice (3rd edition) , 2010 .

[23]  D. Sackett,et al.  Evidence based medicine: what it is and what it isn't , 1996, BMJ.