Reliability and validity of the American Hospital Association's national longitudinal survey of health information technology adoption.

OBJECTIVE To evaluate the internal consistency, construct validity, and criterion validity of a battery of items measuring information technology (IT) adoption, included in the American Hospital Association (AHA) IT Supplement Survey. METHODS We analyzed the 2012 release of the AHA IT Supplement Survey. We performed reliability analysis using Cronbach's α and part-whole correlations, construct validity analysis using principal component analysis (PCA), and criterion validity analysis by assessing the items' sensitivity and specificity of predicting attestation to Medicare Meaningful Use (MU). RESULTS Twenty-eight items of the 31-item instrument and five of six functionality subcategories defined by the AHA all produced reliable scales (α's between 0.833 and 0.958). PCA mostly confirmed the AHA's categorization of functionalities; however, some items loaded only weakly onto the factor most associated with their survey category, and one category loaded onto two separate factors. The battery of items was a valid predictor of attestation to MU, producing a sensitivity of 0.82 and a specificity of 0.72. DISCUSSION The battery of items performed well on most indices of reliability and validity. However, they lack some components of ideal survey design, leaving open the possibility that respondents are not responding independently to each item in the survey. Despite measuring only a portion of the objectives required for attestation to MU, the items are a moderately sensitive and specific predictor of attestation. CONCLUSIONS The analyzed instrument exhibits satisfactory reliability and validity.

[1]  John B. Carroll,et al.  The effect of difficulty and chance success on correlations between items or between tests , 1945 .

[2]  Arnold Zellner,et al.  Introduction to Measurement with Theory , 2009 .

[3]  Maulik S Joshi,et al.  Progress toward meaningful use: hospitals' adoption of electronic health records. , 2011, The American journal of managed care.

[4]  Sowmya R. Rao,et al.  Use of electronic health records in U.S. hospitals. , 2009, The New England journal of medicine.

[5]  Diana Adler,et al.  Using Multivariate Statistics , 2016 .

[6]  M. Diana,et al.  An assessment of Health Care Information and Management Systems Society and Leapfrog data on computerized provider order entry. , 2011, Health services research.

[7]  W. H. Schuur,et al.  Mokken Scale Analysis: Between the Guttman Scale and Parametric Item Response Theory , 2003, Political Analysis.

[8]  Martin Hensher.  Acute Care Hospitals , 2008 .

[9]  Farzad Mostashari,et al.  Adoption of electronic health records grows rapidly, but fewer than half of US hospitals had at least a basic system in 2012. , 2013, Health affairs.

[10]  P. Shekelle,et al.  Systematic Review: Impact of Health Information Technology on Quality, Efficiency, and Costs of Medical Care , 2006, Annals of Internal Medicine.

[11]  Aaron Baird,et al.  Corporate Governance and the Adoption of Health Information Technology Within Integrated Delivery Systems , 2014, Health care management review.

[12]  R. Darlington,et al.  Factor Analysis , 2008 .

[13]  Hhs Centers for Medicare Medicare Services.  Medicare and Medicaid programs; electronic health record incentive program. Final rule. , 2010, Federal register.

[14]  Nir Menachemi,et al.  The agreement and internal consistency of national hospital EMR measures , 2011, Health care management science.

[15]  Charles P Friedman,et al.  Alternative measures of electronic health record adoption among hospitals. , 2010, The American journal of managed care.

[16]  C. DesRoches,et al.  A progress report on electronic health records in U.S. hospitals. , 2010, Health affairs.

[17]  Li Miao,et al.  Health Information Technology Adoption in U.S. Acute Care Hospitals , 2013, Journal of Medical Systems.

[18]  Edward H. Shortliffe,et al.  Evaluation Methods in Biomedical Informatics , 2000 .