Web-Based Collection of Expert Opinion on Routine Scalp EEG: Software Development and Interrater Reliability

Computerized detection of epileptiform transients (ETs), characterized by interictal spikes and sharp waves in the EEG, has been a research goal for the last 40 years. A reliable method for detecting ETs would assist physicians in interpretation and improve efficiency in reviewing long-term EEG recordings. Computer algorithms developed thus far for detecting ETs are not as reliable as human experts, primarily due to the large number of false-positive detections. Comparing the performance of different algorithms is difficult because each study uses individual EEG test datasets. In this article, we present EEGnet, a distributed web-based platform for the acquisition and analysis of large-scale training datasets for comparison of different EEG ET detection algorithms. This software allows EEG scorers to log in through the web, mark EEG segments of interest, and categorize segments of interest using a conventional clinical EEG user interface. This software platform was used by seven board-certified academic epileptologists to score 40 short 30-second EEG segments from 40 patients, half containing ETs and half containing artifacts and normal variants. The software performance was adequate. Interrater reliability for marking the location of paroxysmal activity was low. Interrater reliability of marking artifacts and ETs was high and moderate, respectively.

[1]  Jocelyne Fayn,et al.  Can the lessons learned from the assessment of automated electrocardiogram analysis in the Common Standards for quantitative Electrocardiography study benefit measurement of delayed contrast-enhanced magnetic resonance images? , 2007, Journal of electrocardiology.

[2]  R. J. Ellingson,et al.  On the reliability of clinical EEG interpretation. , 1959, Journal of Nervous and Mental Disease.

[3]  Erasmus Mc,et al.  Interobserver reliability of visual interpretation of electroencephalograms in children with newly diagnosed seizures , 2006, Developmental medicine and child neurology.

[4]  Guideline Four: Standards of Practice in Clinical Electroencephalography , 1986, American journal of electroneurodiagnostic technology.

[5]  Selim R. Benbadis,et al.  Errors in EEGs and the misdiagnosis of epilepsy: Importance, causes, consequences, and proposed remedies , 2007, Epilepsy & Behavior.

[6]  F. Duffy,et al.  Spike detection. I. Correlation and reliability of human experts. , 1996, Electroencephalography and clinical neurophysiology.

[7]  R. Woody.  Intra-judge reliability in clinical electroencephalography. , 1966, Journal of clinical psychology.

[8]  C. V. van Donselaar,et al.  Value of the electroencephalogram in adult patients with untreated idiopathic first seizures. , 1992, Archives of neurology.

[9]  D. Gilbert.  Interobserver reliability of visual interpretation of electroencephalograms in children with newly diagnosed seizures. , 2006, Developmental medicine and child neurology.

[10]  S. Fujimoto,et al.  An intervention to improve the interrater reliability of clinical EEG interpretations , 2003, Psychiatry and clinical neurosciences.

[11]  J. L. Willems,et al.  The Diagnostic Performance of Computer Programs for the Interpretation of Electrocardiograms , 1991 .

[12]  Ronald G. Emerson,et al.  Spike detection II: automatic, perception-based detection and clustering , 1999, Clinical Neurophysiology.

[13]  E Mjolsness,et al.  Machine learning for science: state of the art and future prospects. , 2001, Science.

[14]  A Värri,et al.  A simple format for exchange of digitized polygraphic recordings. , 1992, Electroencephalography and clinical neurophysiology.

[15]  Jonathan J. Halford,et al.  Computerized epileptiform transient detection in the scalp electroencephalogram: Obstacles to progress and the example of computerized ECG interpretation , 2009, Clinical Neurophysiology.

[16]  J. L. Willems,et al.  The diagnostic performance of computer programs for the interpretation of electrocardiograms. , 1992, The New England journal of medicine.

[17]  S. Little,et al.  INTRA‐RATER RELIABILITY OF EEG INTERPRETATIONS , 1962, The Journal of nervous and mental disease.

[18]  Martin A. Green,et al.  Reliability of Clinical Interpretation of the Electroencephalogram , 1975 .

[19]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[20]  N. Fountain,et al.  EEG Is an Essential Clinical Tool: Pro and Con , 2006, Epilepsia.