Assessing Exposure Misclassification by Expert Assessment in Multicenter Occupational Studies

Background: In a multicenter case-control study of lung cancer in central and eastern Europe and in Liverpool, exposure to occupational agents was assessed by teams of local experts. We performed an interteam agreement study to estimate the levels of exposure misclassification and the expected attenuation of the risk estimate. Methods: Eight teams of experts and a reference rater assessed exposure to 70 putative lung carcinogens for 19 jobs. Agreement among teams was calculated through Cohen’s kappa, sensitivity, and specificity. Results: Each team showed an overall fair to good agreement with the reference (kappa between 0.53 and 0.64). The agreement among teams in the presence of exposure was excellent for 9 agents, fair to good for 16, and poor for 29. For all agents the specificity was high (average 0.94), although sensitivity varied considerably. Conclusions: This study of expert exposure assessment showed a small range in reliability among teams of experts, but large differences among agents. This paper presents the range in levels of misclassification that can be expected using experts for assessing occupational exposure to different agents, and the attenuation of the odds ratio that can be expected to result from this misclassification.

[1]  D. Hémon,et al.  Retrospective evaluation of the exposure to polycyclic aromatic hydrocarbons: comparative assessments with a job exposure matrix and by experts in industrial hygiene. , 1993, International journal of epidemiology.

[2]  John S. Evans,et al.  Subjective Estimation of Toluene Exposures: A Calibration Study of Industrial Hygienists , 1989 .

[3]  H. Kromhout,et al.  Semiquantitative Estimates of Exposure to Methylene Chloride and Styrene: The Influence of Quantitative Exposure Data , 1991 .

[4]  S. Semple,et al.  A training exercise in subjectively estimating inhalation exposures. , 2001, Scandinavian journal of work, environment & health.

[5]  J. Trape,et al.  Rapid decline in child mortality in a rural area of Senegal. , 1993, International journal of epidemiology.

[6]  P. Vineis,et al.  Inter-rater agreement in the assessment of occupational exposure to herbicides. , 1988, La Medicina del lavoro.

[7]  J. Mcdonald,et al.  Psychiatric disorders and occupational exposure to solvents. , 1992, British journal of industrial medicine.

[8]  J Siemiatycki,et al.  Self-assessed versus expert-assessed occupational exposures. , 1996, American journal of epidemiology.

[9]  F Berrino,et al.  Retrospective evaluation of occupational exposure to organic solvents: questionnaire and job exposure matrix. , 1993, International journal of epidemiology.

[10]  J Siemiatycki,et al.  Inter-rater agreement in assessing occupational exposure in a case-control study. , 1986, British journal of industrial medicine.

[11]  F Berrino,et al.  Sources of discrepancies between a job exposure matrix and a case by case expert assessment for occupational exposure to formaldehyde and wood-dust. , 1993, International journal of epidemiology.

[12]  C C Johnson,et al.  Comparability of different methods of retrospective exposure assessment of metals in manufacturing industries. , 1997, American journal of industrial medicine.

[13]  J Siemiatycki,et al.  Costs and statistical power associated with five methods of collecting occupation exposure information for population-based case-control studies. , 1989, American journal of epidemiology.

[14]  Q. Mcnemar.  Note on the sampling error of the difference between correlated proportions or percentages , 1947, Psychometrika.

[15]  P A Stewart,et al.  Comparison of industrial hygienists' exposure evaluations for an epidemiologic study. , 2000, Scandinavian journal of work, environment & health.

[16]  A. E. Maxwell.  Comparing the Classification of Subjects by Two Independent Judges , 1970, British Journal of Psychiatry.

[17]  T D Koepsell,et al.  Assessment of occupational exposures in community-based case-control studies. , 1998, Annual review of public health.

[18]  R. McNamee.  Retrospective assessment of occupational exposure to hydrocarbons: job exposure matrices vs expert evaluation of questionnaires , 1996 .

[19]  A. Olshan,et al.  Occupational exposure assessment in case–control studies: opportunities for improvement , 2002, Occupational and environmental medicine.

[20]  J. Carlin,et al.  Bias, prevalence and kappa. , 1993, Journal of clinical epidemiology.

[21]  H Kromhout,et al.  Assessment of occupational exposures in a general population: comparison of different methods. , 1999, Occupational and environmental medicine.

[22]  J. Siemiatycki,et al.  Reliability of an expert rating procedure for retrospective assessment of occupational exposures in community-based case-control studies. , 1997, American journal of industrial medicine.

[23]  A. Feinstein,et al.  High agreement but low kappa: I. The problems of two paradoxes. , 1990, Journal of clinical epidemiology.

[24]  P. Stewart.  Challenges to retrospective exposure assessment. , 1999, Scandinavian journal of work, environment & health.

[25]  M Macaluso,et al.  Inter-rater agreement in the assessment of solvent exposure at a car assembly plant. , 1993, American Industrial Hygiene Association journal.

[26]  G. Belle,et al.  Occupational exposures and amyotrophic lateral sclerosis. A population-based case-control study. , 1997, American journal of epidemiology.

[27]  G. Benke,et al.  Retrospective assessment of occupational exposure to chemicals in community-based studies: validity and repeatability of industrial hygiene panel ratings. , 1997, International journal of epidemiology.

[28]  Walter F. Stewart,et al.  False Positive Exposure Errors and Low Exposure Prevalence in Community-Based Case-Control Studies , 1991 .

[29]  H. Kromhout,et al.  Agreement between qualitative exposure estimates and quantitative exposure measurements. , 1987, American journal of industrial medicine.

[30]  M. Pike,et al.  An improved approximate formula for calculating sample sizes for comparing two binomial distributions. , 1978, Biometrics.

[31]  C. Hertzman,et al.  A comparison of exposure estimates by worker raters and industrial hygienists. , 1989, Scandinavian journal of work, environment & health.