Electronic Quality of Life Assessment Using Computer-Adaptive Testing

Background Quality of life (QoL) questionnaires are desirable for clinical practice but can be time-consuming to administer and interpret, making their widespread adoption difficult. Objective Our aim was to assess the performance of the World Health Organization Quality of Life (WHOQOL)-100 questionnaire as four item banks to facilitate adaptive testing using simulated computer adaptive tests (CATs) for physical, psychological, social, and environmental QoL. Methods We used data from the UK WHOQOL-100 questionnaire (N=320) to calibrate item banks using item response theory, which included psychometric assessments of differential item functioning, local dependency, unidimensionality, and reliability. We simulated CATs to assess the number of items administered before prespecified levels of reliability was met. Results The item banks (40 items) all displayed good model fit (P>.01) and were unidimensional (fewer than 5% of t tests significant), reliable (Person Separation Index>.70), and free from differential item functioning (no significant analysis of variance interaction) or local dependency (residual correlations < +.20). When matched for reliability, the item banks were between 45% and 75% shorter than paper-based WHOQOL measures. Across the four domains, a high standard of reliability (alpha>.90) could be gained with a median of 9 items. Conclusions Using CAT, simulated assessments were as reliable as paper-based forms of the WHOQOL with a fraction of the number of items. These properties suggest that these item banks are suitable for computerized adaptive assessment. These item banks have the potential for international development using existing alternative language versions of the WHOQOL items.

[1]  P. Callaghan,et al.  Evaluating and Quantifying User and Carer Involvement in Mental Health Care Planning (EQUIP): Co-Development of a New Patient-Reported Outcome Measure , 2016, PloS one.

[2]  M. Groenvold,et al.  An emotional functioning item bank of 24 items for computerized adaptive testing (CAT) was established. , 2016, Journal of clinical epidemiology.

[3]  S. Skevington,et al.  Evaluating a new methodology for providing individualized feedback in healthcare on quality of life and its importance, using the WHOQOL-BREF in a community population , 2016, Quality of Life Research.

[4]  Kathleen Scalise,et al.  Use of open-source software for adaptive measurement: Concerto as an R-based computer adaptive development and delivery platform. , 2015, The British journal of mathematical and statistical psychology.

[5]  A. Astrup,et al.  A Randomized, Controlled Trial of 3.0 mg of Liraglutide in Weight Management. , 2015, The New England journal of medicine.

[6]  Craig K Enders,et al.  Including auxiliary item information in longitudinal data analyses improved handling missing questionnaire outcome data. , 2015, Journal of clinical epidemiology.

[7]  P. Bower,et al.  Routine provision of information on patient‐reported outcome measures to healthcare providers and patients in clinical practice , 2015 .

[8]  S. Skevington,et al.  Using guided individualised feedback to review self-reported quality of life in health and its importance , 2015, Psychology & health.

[9]  R. Siegert,et al.  Using feedback from patient-reported outcome measures in mental health services: a scoping study and typology. , 2015, Psychiatric services.

[10]  Chris Feudtner,et al.  Improving the care of children with advanced cancer by using an electronic patient-reported feedback intervention: results from the PediQUEST randomized controlled trial. , 2014, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[11]  P. Bower,et al.  Development of a Multimorbidity Illness Perceptions Scale (MULTIPleS) , 2013, PloS one.

[12]  Jonathan Cavanagh,et al.  Challenges and Implications of Routine Depression Screening for Depression in Chronic Disease and Multimorbidity: A Cross Sectional Study , 2013, PloS one.

[13]  A. Tennant,et al.  Rasch analysis of the WHOQOL-BREF in post polio syndrome. , 2013, Journal of rehabilitation medicine.

[14]  N. Aaronson,et al.  The EORTC computer-adaptive tests measuring physical functioning and fatigue exhibited high levels of measurement precision and efficiency. , 2013, Journal of clinical epidemiology.

[15]  K. Sijtsma,et al.  Dimensionality of the Hospital Anxiety and Depression Scale (HADS) in Cardiac Patients , 2012, Assessment.

[16]  R. Philip Chalmers,et al.  mirt: A Multidimensional Item Response Theory Package for the R Environment , 2012 .

[17]  van der Ark,et al.  New Developments in Mokken Scale Analysis in R , 2012 .

[18]  S. Skevington,et al.  Expecting a good quality of life in health: assessing people with diverse diseases and conditions using the WHOQOL‐BREF , 2012, Health expectations : an international journal of public participation in health care and health policy.

[19]  M. Power,et al.  Cross-Cultural Evaluation of the WHOQOL-BREF Domains in Primary Care Depressed Patients Using Rasch Analysis , 2012, Medical decision making : an international journal of the Society for Medical Decision Making.

[20]  A. Tennant,et al.  Rasch analysis of the hospital anxiety and depression scale (hads) for use in motor neurone disease , 2011, Health and quality of life outcomes.

[21]  P. Cuijpers,et al.  Applying computerized adaptive testing to the CES-D scale: A simulation study , 2011, Psychiatry Research.

[22]  Wijbrandt H van Schuur,et al.  Ordinal Item Response Theory: Mokken Scale Analysis , 2011 .

[23]  P. Selby,et al.  Patients report improvements in continuity of care when quality of life assessments are used routinely in oncology practice: secondary outcomes of a randomised controlled trial. , 2010, European journal of cancer.

[24]  D. Amtmann,et al.  Development of a PROMIS item bank to measure pain interference , 2010, PAIN.

[25]  A. Carr,et al.  The routine use of patient reported outcome measures in healthcare settings , 2010, BMJ : British Medical Journal.

[26]  M. Hackshaw Association of patient-reported outcomes with progression-free survival in malignant pleural mesothelioma , 2010 .

[27]  S. Reise,et al.  Efficiency of static and computer adaptive short forms compared to full-length measures of depressive symptoms , 2010, Quality of Life Research.

[28]  Seung W. Choi,et al.  Firestar: Computerized Adaptive Testing Simulation Program for Polytomous Item Response Theory Models , 2009 .

[29]  Seung W Choi,et al.  Comparison of CAT Item Selection Criteria for Polytomous Items , 2009, Applied psychological measurement.

[30]  David Cella,et al.  Progress in Assessing Physical Function in Arthritis: PROMIS Short Forms and Computerized Adaptive Testing , 2009, The Journal of Rheumatology.

[31]  Peter Bower,et al.  The GP Patient Survey for use in primary care in the National Health Service in the UK – development and psychometric characteristics , 2009, BMC family practice.

[32]  A. Jette,et al.  A computer-adaptive disability instrument for lower extremity osteoarthritis research demonstrated promising breadth, precision, and reliability. , 2009, Journal of clinical epidemiology.

[33]  Tilo Kircher,et al.  Development of an item bank for the assessment of depression in persons with mental illnesses and physical diseases using Rasch analysis. , 2009, Rehabilitation psychology.

[34]  G. Engelhard Historical Perspectives on Invariant Measurement: Guttman, Rasch, and Mokken , 2008 .

[35]  K. Sijtsma,et al.  A Comparative Study of the Dimensionality of the Self-Concealment Scale Using Principal Components Analysis and Mokken Scale Analysis , 2008, Journal of Personality Assessment.

[36]  G. Guyatt,et al.  The impact of measuring patient-reported outcomes in clinical practice: a systematic review of the literature , 2008, Quality of Life Research.

[37]  David J Weiss,et al.  Psychometric Evaluation and Calibration of Health-Related Quality of Life Item Banks: Plans for the Patient-Reported Outcomes Measurement Information System (PROMIS) , 2007, Medical care.

[38]  R. Gershon,et al.  The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment , 2007, Quality of Life Research.

[39]  Julie F Pallant,et al.  An introduction to the Rasch measurement model: an example using the Hospital Anxiety and Depression Scale (HADS). , 2007, The British journal of clinical psychology.

[40]  van der Ark,et al.  Mokken Scale Analysis in R , 2007 .

[41]  Dimitris Rizopoulos,et al.  ltm: An R Package for Latent Variable Modeling and Item Response Analysis , 2006 .

[42]  R. Fitzpatrick,et al.  Impact of patient-reported outcome measures on routine practice: a structured review. , 2006, Journal of evaluation in clinical practice.

[43]  M. Power,et al.  The EUROHIS-QOL 8-item index: psychometric results of a cross-cultural field study. , 2006, European journal of public health.

[44]  S. Gates,et al.  Maximising response to postal questionnaires – A systematic review of randomised trials in health research , 2006, BMC medical research methodology.

[45]  Dimitrios Rizopoulos ltm: An R Package for Latent Variable Modeling and Item Response Theory Analyses , 2006 .

[46]  How much do doctors use quality of life information in primary care? Testing the Trans-Theoretical Model of behaviour change , 2005, Quality of Life Research.

[47]  R. Meijer,et al.  Analyzing psychopathology items: a case for nonparametric item response theory modeling. , 2004, Psychological methods.

[48]  J. Hibbard,et al.  Development of the Patient Activation Measure (PAM): conceptualizing and measuring activation in patients and consumers. , 2004, Health services research.

[49]  S. Skevington,et al.  The World Health Organization's WHOQOL-BREF quality of life assessment: Psychometric properties and results of the international field trial. A Report from the WHOQOL Group , 2004, Quality of Life Research.

[50]  Galina Velikova,et al.  Measuring quality of life in routine oncology practice improves communication and patient well-being: a randomized controlled trial. , 2004, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[51]  J. Thonnard,et al.  Assessing and Adjusting for Cross-Cultural Validity of Impairment and Activity Limitation Scales Through Differential Item Functioning Within the Framework of the Rasch Model: The PRO-ESOR Project , 2004, Medical care.

[52]  David Andrich,et al.  Controversy and the Rasch Model: A Characteristic of Incompatible Paradigms? , 2004, Medical care.

[53]  David Cella,et al.  Item banking to improve, shorten and computerize self-reported fatigue: An illustration of steps to create a core item bank from the FACIT-Fatigue Scale , 2003, Quality of Life Research.

[54]  W. H. Schuur,et al.  Mokken Scale Analysis: Between the Guttman Scale and Parametric Item Response Theory , 2003, Political Analysis.

[55]  M. Stewart Towards a global definition of patient centred care , 2001, BMJ : British Medical Journal.

[56]  G. Karabatsos,et al.  The Rasch model, additive conjoint measurement, and new models of probabilistic measurement theory. , 2001, Journal of applied measurement.

[57]  A Leplege,et al.  Methodological issues in using the Rasch model to select cross culturally equivalent items in order to develop a Quality of Life index: the analysis of four WHOQOL-100 data sets (Argentina, France, Hong Kong, United Kingdom). , 2000, Journal of applied measurement.

[58]  D. Roter The enduring and evolving nature of the patient-physician relationship. , 2000, Patient education and counseling.

[59]  K. Meadows,et al.  The effectiveness of the use of patient-based measures of health in routine practice in improving the process and outcomes of patient care: a literature review. , 1999, Journal of evaluation in clinical practice.

[60]  S. Skevington Measuring quality of life in Britain: introducing the WHOQOL-100. , 1999, Journal of psychosomatic research.

[61]  N. Aaronson Assessing quality of life in clinical practice in oncology , 1999 .

[62]  M. Power,et al.  Development of the World Health Organization WHOQOL-BREF Quality of Life Assessment , 1998, Psychological Medicine.

[63]  S. Saxena,et al.  The World Health Organization Quality of Life Assessment (WHOQOL): development and general psychometric properties. , 1998, Social science & medicine.

[64]  S. Saxena,et al.  The World Health Organization Quality of Life assessment (WHOQOL): position paper from the World Health Organization. , 1995, Social science & medicine.

[65]  Gregory Camilli Teacher’s Corner: Origin of the Scaling Constant d = 1.7 in Item Response Theory , 1994 .

[66]  Gregory Camilli,et al.  Origin of the scaling constant d = 1.7 in item response theory. , 1994 .

[67]  J. Linacre,et al.  Sample size and item calibration stability , 1994 .

[68]  Howard Wainer,et al.  Item Clusters and Computerized Adaptive Testing: A Case for Testlets , 1987 .

[69]  L. Crocker,et al.  Introduction to Classical and Modern Test Theory , 1986 .

[70]  G. Masters A rasch model for partial credit scoring , 1982 .

[71]  Howard Wainer,et al.  The Rasch Model as Additive Conjoint Measurement , 1979 .

[72]  R. J. Mokken,et al.  A Theory and Procedure of Scale Analysis: With Applications in Political Research , 1971 .

[73]  R. Luce,et al.  Simultaneous conjoint measurement: A new type of fundamental measurement , 1964 .

[74]  J. Loevinger A systematic approach to the construction and evaluation of tests of ability. , 1947 .