Simulation study of activities of daily living functions using online computerized adaptive testing

BackgroundComputer adaptive testing (CAT) of the activities of daily living (ADL) functions is required (i) to reveal the advantages of using an efficient and accurate estimation method, (ii) to determine the cutpoint for classifying ADL strata in patients with stroke, and (iii) to evaluate the feasibility of online CAT used in clinical settings for smartphones.MethodsNormally standardized distributions of ADL measurements were simulated using item parameters from published papers. We retrieved item parameters of the combined Barthel Index and Frenchay Activities Index from the literature (the 23-item comprehensive ADL [CADL] and 34-item ADL scales) and simulated three 1000-person measures from a normal standard CAT distribution: [i] CADL (CADL-CAT), [ii] ADL (ADL-CAT), and [iii] NAT (Non-Adaptive Testing). The cutpoints of ADL person strata were determined using a norm-reference method. Maximum a posteriori estimation, expected a posteriori estimation, and maximum likelihood estimation (MAP) were used to compare the Pearson correlation coefficients and different number ratios of paired measures yielded by CAT and NAT. The number of items and the cutpoints for the scale were separately determined.ResultsWe found that (i) correlation coefficients for the three CAT-estimated measures were 0.77 (CADL), 0.93 (Male ADL), and 0.93 (Female ADL) compared with their NAT counterparts. Different number ratios of person-paired measures between CAT and NAT for the three scales were all less than 5 %, indicating no difference exists between CAT and NAT. However, CAT might be 66 % more efficient than NAT. (ii) The estimated cutpoints of T scores (i.e., with a mean of 50 and a standard deviation of 10) were 45, 55, and 65 (e.g., separating person ADL function to four strata with not active, fairly active, active, and very active). (iii) An available-for-download online ADL-CAT APP for clinical practice was demonstrated.ConclusionsAn online ADL-CAT APP using the MAP method was created and used on smartphones to classify ADL strata in patients with stroke.

[1]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[2]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .

[3]  A. Jette,et al.  Functional disability assessment. , 1987, Physical therapy.

[4]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[5]  A cross-validation of the comprehensive assessment of activities of daily living after stroke. , 1999, Scandinavian journal of rehabilitation medicine.

[6]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[7]  D. D. De Beurs,et al.  Applying Computer Adaptive Testing to Optimize Online Assessment of Suicidal Behavior: A Simulation Study , 2014, Journal of medical Internet research.

[8]  John M Linacre,et al.  Optimizing rating scale category effectiveness. , 2002, Journal of applied measurement.

[9]  O. Shechtman,et al.  Development of an item bank for a computerised adaptive test of upper-extremity function , 2011, Disability and rehabilitation.

[10]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[11]  J. Jeng,et al.  Comparison of the psychometric characteristics of the functional independence measure, 5 item Barthel index, and 10 item Barthel index in patients with stroke , 2002, Journal of neurology, neurosurgery, and psychiatry.

[12]  Terry Davis,et al.  The role of health literacy in patient-physician communication. , 2002, Family medicine.

[13]  T. G. Rhee,et al.  Cancer literacy as a mediator for cancer screening behaviour in Korean adults. , 2016, Health & social care in the community.

[14]  Michael R. Harwell,et al.  Monte Carlo Studies in Item Response Theory , 1996 .

[15]  A. Rush,et al.  Methods to improve diagnostic accuracy in a community mental health setting. , 2000, The American journal of psychiatry.

[16]  Shih-Chung Chen,et al.  Web-based computer adaptive assessment of individual perceptions of job satisfaction for hospital workplace employees , 2011, BMC medical research methodology.

[17]  S. Embretson,et al.  Item response theory for psychologists , 2000 .

[18]  Wen-Chung Wang,et al.  Online assessment of patients' views on hospital performances using Rasch model's KIDMAP diagram , 2009, BMC health services research.

[19]  Ching-Fan Sheu,et al.  Psychometric properties of the Berg Balance Scale in a community-dwelling elderly resident population in Taiwan. , 2006, Journal of the Formosan Medical Association = Taiwan yi zhi.

[20]  Joanna Bokovoy,et al.  Carts and care , 2007, Nursing management.

[21]  Klaus Jung,et al.  Comparison of eight computer programs for receiver-operating characteristic analysis. , 2003, Clinical chemistry.

[22]  Evaluation of stroke patients with the extended activities of daily living scale in Taiwan , 2000, Disability and rehabilitation.

[23]  Kyung T. Han,et al.  WinGen: Windows Software That Generates Item Response Theory Parameters and Item Responses , 2007 .

[24]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[25]  Hae-Ra Han,et al.  Potential links between health literacy and cervical cancer screening behaviors: a systematic review , 2016, Psycho-oncology.

[26]  I. Hsueh,et al.  Rasch Analysis of Combining Two Indices to Assess Comprehensive ADL Function in Stroke Patients , 2004, Stroke.

[27]  J. Ware,et al.  Involving consumers in quality of care assessment. , 1988, Health affairs.

[28]  A. Tennant,et al.  An application of computerised adaptive testing for measuring health status in patients with knee osteoarthritis , 2010, Disability and rehabilitation.

[29]  G. Andersson,et al.  Multimedia Appendix 1 , 2011 .

[30]  T. Chien,et al.  Applying Computerized Adaptive Testing to the Negative Acts Questionnaire-Revised: Rasch Analysis of Workplace Bullying , 2014, Journal of medical Internet research.

[31]  Kate Lorig,et al.  Internet Versus Mailed Questionnaires: A Randomized Comparison , 2004, Journal of medical Internet research.

[32]  Virginia P. Richmond,et al.  Reliability and Separation of Factors on the Assertiveness-Responsiveness Measure , 1990 .

[33]  How-Ran Guo,et al.  Bmc Medical Research Methodology Open Access Kidmap, a Web Based System for Gathering Patients' Feedback on Their Doctors , 2009 .

[34]  E. Muraki.  A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[35]  Dorothy T. Thayer,et al.  Differential Item Performance and the Mantel-Haenszel Procedure. , 1986 .

[36]  T. Olsen,et al.  Comprehensive assessment of activities of daily living in stroke. The Copenhagen Stroke Study. , 1997, Archives of physical medicine and rehabilitation.

[37]  M. Mullee,et al.  Validation of the Frenchay Activities Index in a general population aged 16 years and older. , 2000, Archives of physical medicine and rehabilitation.

[38]  H. Swartz,et al.  Diagnosis of nonpsychotic patients in community clinics. , 2000, The American journal of psychiatry.

[39]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[40]  D. Amtmann,et al.  Developing a contemporary patient-reported outcomes measure for spinal cord injury. , 2011, Archives of physical medicine and rehabilitation.

[41]  Stephanie J Mitchell,et al.  Internet and Mobile Technology Use Among Urban African American Parents: Survey Study of a Clinical Population , 2012, Journal of medical Internet research.

[42]  M. Limburg,et al.  The Frenchay Activities Index: Assessment of Functional Status in Stroke Patients , 1993, Stroke.

[43]  Wen-Chung Wang,et al.  Development of a Computerized Adaptive Test for Assessing Balance Function in Patients With Stroke , 2010, Physical Therapy.

[44]  Jonathan B. Singer,et al.  Screening for Anxiety and Depression in Community Mental Health: The Beck Anxiety and Depression Inventories , 2008, Community Mental Health Journal.

[45]  P. Selby,et al.  A Rasch and factor analysis of the Functional Assessment of Cancer Therapy-General (FACT-G) , 2007, Health and quality of life outcomes.

[46]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[47]  R. Rosenfeld.  Patients , 2012, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[48]  R. M. Smith,et al.  Fit analysis in latent trait measurement models. , 2000, Journal of applied measurement.

[49]  A. Dowson,et al.  Applications of computerized adaptive testing (CAT) to the assessment of headache impact , 2003, Quality of Life Research.

[50]  R. D. Bock,et al.  Adaptive EAP Estimation of Ability in a Microcomputer Environment , 1982 .

[51]  B. Wright,et al.  Observations are always ordinal; measurements, however, must be interval. , 1989, Archives of physical medicine and rehabilitation.

[52]  W. Spector,et al.  Combining activities of daily living with instrumental activities of daily living to measure functional disability. , 1998, The journals of gerontology. Series B, Psychological sciences and social sciences.

[53]  F. Mahoney,et al.  FUNCTIONAL EVALUATION: THE BARTHEL INDEX. , 2018, Maryland state medical journal.

[54]  D. Andrich.  A rating formulation for ordered response categories , 1978 .

[55]  Y. Wang,et al.  Utilizing Rasch measurement models to develop a computer adaptive self-report of walking, climbing, and running , 2008, Disability and rehabilitation.

[56]  C. Skilbeck,et al.  An activities index for use with stroke patients. , 1983, Age and ageing.

[57]  Chun-Hou Wang,et al.  Development of a Computerized Adaptive Test for Assessing Activities of Daily Living in Outpatients With Stroke , 2013, Physical Therapy.

[58]  Ian M. Bennett,et al.  Understanding the Health Literacy of America: Results of the National Assessment of Adult Literacy , 2009, Orthopedic nursing.

[59]  K. Huybrechts,et al.  The Barthel Index and modified Rankin Scale as prognostic tools for long-term outcomes after stroke: a qualitative review of the literature , 2007, Current medical research and opinion.

[60]  A. Tennant,et al.  An initial application of computerized adaptive testing (CAT) for measuring disability in patients with low back pain , 2008, BMC musculoskeletal disorders.

[61]  L. Aday,et al.  Designing and conducting health surveys : a comprehensive guide , 2006 .

[62]  Everett V. Smith.  Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. , 2002, Journal of applied measurement.

[63]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[64]  David J. Weiss,et al.  Improving Measurement Quality and Efficiency with Adaptive Testing , 1982 .

[65]  Wen-Chung Wang,et al.  Reliability of 95% confidence interval revealed by expected quality-of-life scores: an example of nasopharyngeal carcinoma patients after radiotherapy using EORTC QLQ-C 30 , 2010, Health and quality of life outcomes.

[66]  S. Paunonen,et al.  A Monte Carlo Comparison of Item and Person Statistics Based on Item Response Theory versus Classical Test Theory , 2002 .

[67]  Melissa S. Yale,et al.  Differential Item Functioning , 2014 .