Test-retest reliability in a computer-based medical history

The authors developed a computer-based medical history for patients to take in their homes via the internet. The history consists of 232 'primary' questions asked of all patients, together with more than 6000 questions, explanations, and suggestions that are available for presentation as determined by a patient's responses. The purpose of this research was to measure the test-retest reliability of the 215 primary questions that have preformatted, mutually exclusive responses of 'Yes,' 'No,' 'Uncertain (Don't know, Maybe),' 'Don't understand,' and 'I'd rather not answer.' From randomly selected patients of doctors affiliated with Beth Israel Deaconess Medical Center in Boston, 48 patients took the history twice with intervals between sessions ranging from 1 to 35 days (mean 7 days; median 5 days). High levels of test-retest reliability were found for most of the questions, but as a result of this study the authors revised five questions. They recommend that structured medical history questions that will be asked of many patients be measured for test-retest reliability before they are put into widespread clinical practice.

[1]  D. Gustafson,et al.  A computer interview for suicide-risk prediction. , 1973, The American journal of psychiatry.

[2]  J. Hubble,et al.  Test-retest reliability of an epidemiological instrument for Parkinson's disease. , 2000, Journal of clinical epidemiology.

[3]  C Safran,et al.  Computer-based interview for screening blood donors for risk of HIV transmission. , 1990, JAMA.

[4]  W. Slack.  Patient-computer dialogue: a hope for the future. , 2010, Mayo Clinic proceedings.

[5]  P. D. Oldham,et al.  Observers' errors in taking medical histories. , 1951, Lancet.

[6]  Christina N. Lessov-Schlaggar,et al.  Test-Retest Reliability of Web-Based Retrospective Self-Report of Tobacco Exposure and Risk , 2009, Journal of medical Internet research.

[7]  W V Slack,et al.  Patient reaction to computer-based medical interviewing. , 1968, Computers and biomedical research, an international journal.

[8]  Jacob Cohen,et al.  Weighted kappa: Nominal scale agreement provision for scaled disagreement or partial credit. , 1968 .

[9]  J. Ford,et al.  The test-retest reliability and concurrent validity of the Subjective Complaints Questionnaire for low back pain. , 2009, Manual therapy.

[10]  R. W. Lucas,et al.  Computer interrogation of patients. , 1976, British medical journal.

[11]  W V Slack,et al.  A history of computerized medical interviews. , 1984, M.D. computing : computers in medical practice.

[12]  Q. Mcnemar.  Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[13]  Warner V. Slack,et al.  Talking to a computer about emotional problems: A comparative study. , 1977 .

[14]  W V Slack,et al.  Patient-computer dialogue. , 1972, The New England journal of medicine.

[15]  L. Mucci,et al.  Test-retest reliability of colorectal testing questions on the Massachusetts Behavioral Risk Factor Surveillance System (BRFSS). , 2005, Preventive medicine.

[16]  J. Fortenberry,et al.  Patient and staff perspectives on the use of a computer counseling tool for HIV and sexually transmitted infection risk reduction. , 2007, The Journal of adolescent health : official publication of the Society for Adolescent Medicine.

[17]  D. Streiner,et al.  Health Measurement Scales: A practical guide to thier development and use , 1989 .

[18]  A. Thiel,et al.  Test-retest reliability of the Eating Disorder Inventory 2. , 2006, Journal of psychosomatic research.

[19]  H. Wolff,et al.  The Cornell medical index; a adjunct to medical interview. , 1949, Journal of the American Medical Association.

[20]  P. Bolton,et al.  The Childhood Asperger Syndrome Test (CAST) , 2007, Autism : the international journal of research and practice.

[21]  E. Waclawski.  Health Measurement Scales—A Practical Guide to Their Development and Use , 2010 .

[22]  R. Kahn,et al.  The Dynamics of Interviewing , 1957 .

[23]  Warner V. Slack.  Cybermedicine: How Computing Empowers Doctors and Patients for Better Health Care , 1997 .

[24]  W V Slack,et al.  Patient-Computer Dialogue: A Review, , 2000, Yearbook of Medical Informatics.

[25]  M. Posternak,et al.  Assessing Past Treatment History: Test-Retest Reliability of the Treatment Response to Antidepressant Questionnaire , 2004, The Journal of nervous and mental disease.

[26]  Steven C Adamson,et al.  Pilot study of providing online care in a primary care setting. , 2010, Mayo Clinic proceedings.

[27]  Warner V Slack,et al.  A 67-year-old man who e-mails his physician. , 2004, JAMA.

[28]  Robert L. Kahn,et al.  The dynamics of interviewing; theory, technique, and cases , 1958 .

[29]  W V Slack,et al.  Relation between age, education, and time to respond to questions in a computer-based medical interview. , 1988, Computers and biomedical research, an international journal.

[30]  J. Bachman,et al.  The patient-computer interview: a neglected tool that can aid the clinician. , 2003, Mayo Clinic proceedings.

[31]  S. M. Rogers,et al.  Adolescent sexual behavior, drug use, and violence: increased reporting with computer survey technology. , 1998, Science.

[32]  J G Mayne,et al.  Toward automating the medical history. , 1968, Mayo Clinic proceedings.

[33]  F. Jelovsek.  Computers in the examining room, the delivery room, and the nursery. , 1984, M.D. computing : computers in medical practice.

[34]  M F Collen,et al.  Reliability of a self-administered medical questionnaire. , 1969, Archives of internal medicine.

[35]  J. R. Landis,et al.  The measurement of observer agreement for categorical data. , 1977, Biometrics.

[36]  W. Slack,et al.  A computer-based medical-history system. , 1966, The New England journal of medicine.

[37]  Patrick J Heagerty,et al.  A Comparison Between Audio Computer-Assisted Self-Interviews and Clinician Interviews for Obtaining the Sexual History , 2004, Sexually transmitted diseases.

[38]  K Y Liang,et al.  Longitudinal data analysis for discrete and continuous outcomes. , 1986, Biometrics.