Training in data definitions improves quality of intensive care data

BackgroundOur aim was to assess the contribution of training in data definitions and data extraction guidelines to improving quality of data for use in intensive care scoring systems such as the Acute Physiology and Chronic Health Evaluation (APACHE) II and Simplified Acute Physiology Score (SAPS) II in the Dutch National Intensive Care Evaluation (NICE) registry.MethodsBefore and after attending a central training programme, a training group of 31 intensive care physicians from Dutch hospitals who were newly participating in the NICE registry extracted data from three sample patient records. The 5-hour training programme provided participants with guidelines for data extraction and strict data definitions. A control group of 10 intensive care physicians, who were trained according the to train-the-trainer principle at least 6 months before the study, extracted the data twice, without specific training in between.ResultsIn the training group the mean percentage of accurate data increased significantly after training for all NICE variables (+7%, 95% confidence interval 5%–10%), for APACHE II variables (+6%, 95% confidence interval 4%–9%) and for SAPS II variables (+4%, 95% confidence interval 1%–6%). The percentage data error due to nonadherence to data definitions decreased by 3.5% after training. Deviations from 'gold standard' SAPS II scores and predicted mortalities decreased significantly after training. Data accuracy in the control group did not change between the two data extractions and was equal to post-training data accuracy in the training group.ConclusionTraining in data definitions and data extraction guidelines is an effective way to improve quality of intensive care scoring data.

[1]  W. Knaus.  The APACHE III Prognostic System , 1992 .

[2]  W. Knaus,et al.  The APACHE III prognostic system. Risk prediction of hospital mortality for critically ill hospitalized adults. , 1991, Chest.

[3]  S Lemeshow,et al.  The Logistic Organ Dysfunction system. A new way to assess organ dysfunction in the intensive care unit. ICU Scoring Group. , 1996, JAMA.

[4]  W J Sibbald,et al.  Interobserver variability in data collection of the APACHE II score in teaching and community hospitals. , 1999, Critical care medicine.

[5]  N. D. de Keizer,et al.  Quality of data collected for severity of illness scores in the Dutch National Intensive Care Evaluation (NICE) registry , 2002, Intensive Care Medicine.

[6]  Stanley Lemeshow,et al.  The Logistic Organ Dysfunction System , 1997 .

[7]  K. Polderman,et al.  Interobserver variability in the use of APACHE II scores , 1999, The Lancet.

[8]  D. Goldhill,et al.  APACHE II, data accuracy and outcome prediction , 1998, Anaesthesia.

[9]  Nicolette de Keizer,et al.  Model Formulation: Defining and Improving Data Quality in Medical Registries: A Literature Review, Case Study, and Generic Framework , 2002, J. Am. Medical Informatics Assoc..

[10]  A. Bersten,et al.  Prospective evaluation of residents and nurses as severity score data collectors , 1992, Critical care medicine.

[11]  P. Landais,et al.  Evaluation of severity scoring systems in ICUs—translation, conversion and definition ambiguities as a source of inter-observer variability in Apache II, SAPS and OSF , 1995, Intensive Care Medicine.

[12]  K. Polderman,et al.  Inter-observer variability in APACHE II scoring: effect of strict guidelines and training , 2001, Intensive Care Medicine.

[13]  Corinne Alberti,et al.  The Logistic Organ Dysfunction system. A new way to assess organ dysfunction in the intensive care unit. ICU Scoring Group. , 1996, JAMA.

[14]  E. Draper,et al.  APACHE II: A severity of disease classification system , 1985, Critical care medicine.

[15]  W. Knaus,et al.  Reliability of a measure of severity of illness: acute physiology of chronic health evaluation--II. , 1992, Journal of clinical epidemiology.

[16]  S. Lemeshow,et al.  A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. , 1993, JAMA.

[17]  S Lemeshow,et al.  Mortality probability models for patients in the intensive care unit for 48 or 72 hours: A prospective, multicenter study , 1994, Critical care medicine.

[18]  K. Polderman,et al.  Intra-observer variability in APACHE II scoring , 2001, Intensive Care Medicine.