Functioning and validity of A Computerized Adaptive Test to measure anxiety (A‐CAT)

Background: The aim of this study was to evaluate the Computerized Adaptive Test to measure anxiety (A‐CAT), a patient‐reported outcome questionnaire that uses computerized adaptive testing to measure anxiety. Methods: The A‐CAT builds on an item bank of 50 items that has been built using conventional item analyses and item response theory analyses. The A‐CAT was administered on Personal Digital Assistants to n=357 patients diagnosed and treated at the department of Psychosomatic Medicine and Psychotherapy, Charité Berlin, Germany. For validation purposes, two subgroups of patients (n=110 and 125) answered the A‐CAT along with established anxiety and depression questionnaires. Results: The A‐CAT was fast to complete (on average in 2 min, 38 s) and a precise item response theory based CAT score (reliability>.9) could be estimated after 4–41 items. On average, the CAT displayed 6 items (SD=4.2). Convergent validity of the A‐CAT was supported by correlations to existing tools (Hospital Anxiety and Depression Scale‐A, Beck Anxiety Inventory, Berliner Stimmungs‐Fragebogen A/D, and State Trait Anxiety Inventory: r=.56–.66); discriminant validity between diagnostic groups was higher for the A‐CAT than for other anxiety measures. Conclusions: The German A‐CAT is an efficient, reliable, and valid tool for assessing anxiety in patients suffering from anxiety disorders and other conditions with significant potential for initial assessment and long‐term treatment monitoring. Future research directions are to explore content balancing of the item selection algorithm of the CAT, to norm the tool to a healthy sample, and to develop practical cutoff scores. Depression and Anxiety, 2008. © 2008 Wiley‐Liss, Inc.

[1]  Mark Kosinski,et al.  Using the SF-12 Health Status Measure to Improve Predictions of Medical Expenditures , 2006, Medical care.

[2]  D. Nease,et al.  Does the severity of mood and anxiety symptoms predict health care utilization? , 1999, The Journal of family practice.

[3]  S. McLachlan,et al.  The application of computer touch-screen technology in screening for psychosocial distress in an ambulatory oncology setting. , 2002, European journal of cancer care.

[4]  J. Ware,et al.  Practical implications of item response theory and computerized adaptive testing: a brief summary of ongoing studies of widely used headache impact scales. , 2000, Medical care.

[5]  M. Kosinski,et al.  Using item response theory to calibrate the Headache Impact Test (HIT™) to the metric of traditional headache scales , 2003, Quality of Life Research.

[6]  V. Starcevic.  Review: worldwide lifetime prevalence of anxiety disorders is 16.6%, with considerable heterogeneity between studies , 2006, Evidence-based mental health.

[7]  J. D. Parker,et al.  Confirmatory factor analysis of the Beck Anxiety and Depression Inventories in patients with major depression. , 1998, Journal of affective disorders.

[8]  M. Hornbrook,et al.  Assessing relative health plan risk with the RAND-36 health survey. , 1995, Inquiry : a journal of medical care organization, provision and financing.

[9]  W W Zung,et al.  A rating instrument for anxiety disorders. , 1971, Psychosomatics.

[10]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[11]  W. Gardner,et al.  Multidimensional Adaptive Testing for Mental Health Problems in Primary Care , 2002, Medical care.

[12]  Donna L Berry,et al.  Usability of a computerized PAINReportIt in the general public with pain and people with cancer pain. , 2003, Journal of pain and symptom management.

[13]  P. Fayers.  Item Response Theory for Psychologists , 2004, Quality of Life Research.

[14]  R. Sitgreaves.  Psychometric theory (2nd ed.). , 1979 .

[15]  Computerized adaptive assessment with the MMPI-2 in a clinical setting. , 1999 .

[16]  S. Reise,et al.  Computerization and Adaptive Administration of the NEO PI-R , 2000, Assessment.

[17]  L. Clark,et al.  Validation of a computerized adaptive version of the Schedule for Nonadaptive and Adaptive Personality (SNAP). , 2005, Psychological assessment.

[18]  Richard N. Jones,et al.  Use of an IRT-based latent variable model to link different forms of the CES-D from the Health and Retirement Study , 2004, Social Psychiatry and Psychiatric Epidemiology.

[19]  C. Schwartz,et al.  Computerized adaptive testing of diabetes impact: A feasibility study of Hispanics and non-Hispanics in an active clinic population , 2006, Quality of Life Research.

[20]  M. Adamek,et al.  Depression and anxiety. , 2013, The Medical journal of Australia.

[21]  A. Mykletun,et al.  Study of anxiety disorder and depression in long-term survivors of testicular cancer. , 2005, Journal of clinical oncology : official journal of the American Society of Clinical Oncology.

[22]  M Rose,et al.  Evaluation of a preliminary physical function item bank supported the expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). , 2008, Journal of clinical epidemiology.

[23]  D J Kupfer,et al.  Exactly what does the Hamilton Depression Rating Scale measure? , 1993, Journal of psychiatric research.

[24]  S. J. Sinclair,et al.  Item Response Theory and Computerized Adaptive Testing: Implications for Outcomes Measurement in Rehabilitation , 2005 .

[25]  M. Bayliss,et al.  A study of the feasibility of Internet administration of a computerized health survey: The headache impact test (HIT™) , 2003, Quality of Life Research.

[26]  R. Bell,et al.  The Beck Anxiety Inventory in a non-clinical sample. , 1995, Behaviour research and therapy.

[27]  S. Mineka,et al.  Comorbidity of anxiety and unipolar mood disorders. , 1998, Annual review of psychology.

[28]  Joanne Greenhalgh,et al.  The use of patient reported outcome measures in routine clinical practice: lack of impact or lack of theory? , 2005, Social science & medicine.

[29]  M. Browne,et al.  Alternative Ways of Assessing Model Fit , 1992 .

[30]  R. Kessler,et al.  Lifetime panic-depression comorbidity in the National Comorbidity Survey. Association with symptoms, impairment, course and help-seeking. , 1998, The British journal of psychiatry : the journal of mental science.

[31]  R A McCormick,et al.  Testing a tripartite model: II. Exploring the symptom structure of anxiety and depression in student, adult, and patient samples. , 1995, Journal of abnormal psychology.

[32]  J. Ruskin,et al.  Psychiatric disorders in medical outpatients complaining of palpitations , 1994, Journal of General Internal Medicine.

[33]  C. Sherbourne,et al.  Prevalence of comorbid anxiety disorders in primary care outpatients. , 1996, Archives of family medicine.

[34]  Daniel J Buysse,et al.  Computerized adaptive measurement of depression: A simulation study , 2004, BMC psychiatry.

[35]  M. Ohayon.  Anxiety disorders: prevalence, comorbidity and outcomes. , 2006, Journal of psychiatric research.

[36]  H. Möller,et al.  Depressive, anxiety, and somatoform disorders in primary care: prevalence and recognition , 2007, Depression and anxiety.

[37]  J Alonso,et al.  Provision of feedback on perceived health status to health care professionals: a systematic review of its impact. , 2000, Medical care.

[38]  Lorena Hsu,et al.  Prevalence and Incidence Studies of Anxiety Disorders: A Systematic Review of the Literature , 2004, Canadian journal of psychiatry. Revue canadienne de psychiatrie.

[39]  Otto B. Walter,et al.  Development of a Computer-adaptive Test for Depression (D-CAT) , 2005, Quality of Life Research.

[40]  Mark Kosinski,et al.  An evaluation of a patient-reported outcomes found computerized adaptive testing was efficient in assessing osteoarthritis impact. , 2006, Journal of clinical epidemiology.

[41]  Richard Balon,et al.  Mood, anxiety, and physical illness: body and mind, or mind and body? , 2006, Depression and anxiety.

[42]  M. Thase,et al.  Differential functioning of the Beck depression inventory in late-life patients: use of item response theory. , 2002, Psychology and aging.

[43]  E. Muraki,et al.  Full-Information Item Factor Analysis , 1988 .

[44]  N. Kar,et al.  Toxoplasma seropositivity and depression: a case report , 2004, BMC psychiatry.

[45]  R. Swinson,et al.  Does the Beck Anxiety Inventory measure anything beyond panic attack symptoms? , 1996, Behaviour research and therapy.

[46]  D. Cooke,et al.  An Item Response Theory Analysis of the Hare Psychopathy Checklist-Revised , 1997 .

[47]  Richard C Gershon,et al.  Computer adaptive testing , 1994 .

[48]  R. Snaith,et al.  The Hospital Anxiety and Depression Scale , 1983 .

[49]  H. Hadjistavropoulos,et al.  Measures of anxiety: is there a difference in their ability to predict functioning at three‐month follow‐up among pain patients? , 2004, European journal of pain.

[50]  J. Butcher,et al.  Current developments and future directions in computerized personality assessment. , 1985, Journal of consulting and clinical psychology.

[51]  David Cella,et al.  Item banking to improve, shorten and computerize self-reported fatigue: An illustration of steps to create a core item bank from the FACIT-Fatigue Scale , 2003, Quality of Life Research.

[52]  L. Carlson,et al.  Computerized Quality-of-Life Screening in a Cancer Pain Clinic , 2001, Journal of palliative care.

[53]  C. Spielberger,et al.  Manual for the State-Trait Anxiety Inventory , 1970 .

[54]  R. Kessler,et al.  Lifetime panic–depression comorbidity in the National Comorbidity Survey , 2000, British Journal of Psychiatry.

[55]  Otto B. Walter,et al.  Entwicklungsschritte für einen computeradaptiven Test zur Erfassung von Angst (A-CAT1) , 2005 .

[56]  F. Porzsolt,et al.  Practical aspects of quality-of-life measurement: design and feasibility study of the quality-of-life recorder and the standardized measurement of quality of life in an outpatient clinic. , 1996, Cancer treatment reviews.

[57]  R. D. Bock,et al.  Adaptive EAP Estimation of Ability in a Microcomputer Environment , 1982 .

[58]  Rob R. Meijer,et al.  Computerized Adaptive Testing: Overview and Introduction , 1999 .

[59]  L. F. Hornke.  Benefits from Computerized Adaptive Testing as Seen in Simulation Studies , 1999 .

[60]  J. Hoyer,et al.  Screening for anxiety in an epidemiological sample: predictive accuracy of questionnaires. , 2002, Journal of anxiety disorders.

[61]  A. Dowson,et al.  Applications of computerized adaptive testing (CAT) to the assessment of headache impact , 2003, Quality of Life Research.

[62]  D H Barlow,et al.  Fear, panic, anxiety, and disorders of emotion. , 1996, Nebraska Symposium on Motivation. Nebraska Symposium on Motivation.

[63]  J F Fries,et al.  The promise of PROMIS: using item response theory to improve assessment of patient-reported outcomes. , 2005, Clinical and experimental rheumatology.

[64]  P. Selby,et al.  Computer-based quality of life questionnaires may contribute to doctor–patient interactions in oncology , 2002, British Journal of Cancer.

[65]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[66]  M. Höfler,et al.  What characteristics of primary anxiety disorders predict subsequent major depressive disorder? , 2004, The Journal of clinical psychiatry.

[67]  K. Blankstein.  The sensation seeker and anxiety reactivity: relationships between the sensation-seeking scales and the activity preference questionnaire. , 1975, Journal of clinical psychology.

[68]  Otto B. Walter,et al.  7 years of experience using Personal Digital Assistants (PDA) for psychometric diagnostics in 6000 inpatients and polyclinic patients , 2002, Mobile Computing in Medicine.

[69]  C. Spielberger.  Manual for the State-Trait Anxiety Inventory (STAI) (Form Y , 1983 .

[70]  R. J. Mokken,et al.  Handbook of modern item response theory , 1997 .

[71]  Y. Ben-Porath,et al.  Computerized adaptive personality testing: a review and illustration with the MMPI-2 Computerized Adaptive Version. , 2007, Psychological assessment.

[72]  George D. Kitas,et al.  Computerized information-gathering in specialist rheumatology clinics: an initial evaluation of an electronic version of the Short Form 36. , 2002, Rheumatology.

[73]  S. Sydeman,et al.  Does generalized anxiety disorder predict coronary heart disease risk factors independently of major depressive disorder? , 2005, Journal of affective disorders.

[74]  James O. Ramsay,et al.  Nonparametric Item Analyses of the Beck Depression Inventory: Evaluating Gender Item Bias and Response Option Weights , 1994 .

[75]  S. Hammond.  An IRT Investigation of the Validity of Non-Patient Analogue Research Using the Beck Depression Inventory , 1995 .

[76]  Anastasia E. Raczek,et al.  Assessing mobility in children using a computer adaptive testing version of the pediatric evaluation of disability inventory. , 2005, Archives of physical medicine and rehabilitation.

[77]  R. MacCallum,et al.  Power analysis and determination of sample size for covariance structure modeling. , 1996 .

[78]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[79]  Otto B. Walter,et al.  Evaluation of a computer‐adaptive test for the assessment of depression (D‐CAT) in clinical application , 2009, International journal of methods in psychiatric research.

[80]  W. Katon,et al.  Collaborative management to achieve treatment guidelines. Impact on depression in primary care. , 1995, JAMA.

[81]  下山 理史,et al.  消化器外科における癌告知の検討 : Hospital Anxiety and Depression Scale(HADS)を用いて , 2002 .

[82]  W. Katon,et al.  Psychiatric illness and irritable bowel syndrome: a comparison with inflammatory bowel disease. , 1990, The American journal of psychiatry.

[83]  W. Zhu.  Test equating: what, why, how? , 1998, Research quarterly for exercise and sport.

[84]  C. Sherbourne,et al.  Summed-score linking using item response theory: application to depression measurement. , 2000, Psychological assessment.

[85]  K. Kobak,et al.  Computer-administered clinical rating scales A review , 1996, Psychopharmacology.

[86]  D. Rice,et al.  Economic costs of anxiety disorders. , 1996, Anxiety.

[87]  Olga V. Demler,et al.  Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. , 2005, Archives of general psychiatry.

[88]  M. Edelen,et al.  Applying item response theory (IRT) modeling to questionnaire development, evaluation, and refinement , 2007, Quality of Life Research.

[89]  M. Stein,et al.  Panic disorder in patients attending a clinic for vestibular disorders. , 1994, The American journal of psychiatry.

[90]  David Cella,et al.  Developing an initial physical function item bank from existing sources. , 2003, Journal of applied measurement.

[91]  Y. Ben-Porath,et al.  Comparability of computerized adaptive and conventional testing with the MMPI-2. , 1991, Journal of personality assessment.

[92]  C. Weems,et al.  Do anxiety and depression cluster into distinct groups?: a test of tripartite model predictions in a community sample of youth , 2006, Depression and anxiety.

[93]  T. Brown,et al.  Current and lifetime comorbidity of the DSM-IV anxiety and mood disorders in a large clinical sample. , 2001, Journal of abnormal psychology.

[94]  S. Haley,et al.  Measuring Physical Function in Patients with Complex Medical and Postsurgical Conditions: A Computer Adaptive Approach , 2005, American journal of physical medicine & rehabilitation.

[95]  B. Reeve,et al.  Item Response Theory and its Applications to Patient-Reported Outcomes Measurement , 2005, Evaluation & the health professions.

[96]  Neil J. Dorans,et al.  Linking scores from multiple health outcome instruments , 2007, Quality of Life Research.