Projected Accuracy Metric for the P300 Speller

The P300 Speller brain-computer interface (BCI) is a virtual keyboard that allows users to type without requiring neuromuscular control. P300 Speller research commonly aims to improve the system accuracy, which is typically estimated by spelling a small number of characters and calculating the percent spelled correctly. In this paper we introduce a new method for estimating the long-term (“projected”) accuracy, which utilizes all available flash data and a probabilistic model of the Speller system to produce an estimate with lower variance and lower granularity than the standard measure. We apply the new method to 110 previously-collected P300 Speller runs to confirm its consistency, and simulate spelling runs from real subject data to demonstrate lower variance on the accuracy estimate for any given amount of data.

[1]  Klaus-Robert Müller,et al.  The BCI competition 2003: progress and perspectives in detection and discrimination of EEG single trials , 2004, IEEE Transactions on Biomedical Engineering.

[2]  Eric W. Sellers,et al.  Predictive Spelling With a P300-Based Brain–Computer Interface: Increasing the Rate of Communication , 2010, Int. J. Hum. Comput. Interact..

[3]  K. A. Colwell,et al.  Bayesian Approach to Dynamically Controlling Data Collection in P300 Spellers , 2013, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[4]  G. Cardarilli,et al.  Performances Evaluation and Optimization of Brain Computer Interface Systems in a Copy Spelling Task , 2007, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[5]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[6]  Nader Pouratian,et al.  Natural language processing with dynamic classification improves P300 speller accuracy and bit rate , 2012, Journal of neural engineering.

[7]  D.J. McFarland,et al.  The wadsworth BCI research and development program: at home with BCI , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[8]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[9]  N. Birbaumer,et al.  An auditory oddball (P300) spelling system for brain-computer interfaces. , 2009, Psychophysiology.

[10]  J. Wolpaw,et al.  A novel P300-based brain–computer interface stimulus presentation paradigm: Moving beyond rows and columns , 2010, Clinical Neurophysiology.

[11]  E. W. Sellers,et al.  Toward enhanced P300 speller performance , 2008, Journal of Neuroscience Methods.

[12]  Dean J Krusienski,et al.  A comparison of classification techniques for the P300 Speller , 2006, Journal of neural engineering.

[13]  G Pfurtscheller,et al.  Estimating the Mutual Information of an EEG-based Brain-Computer Interface , 2002, Biomedizinische Technik. Biomedical engineering.

[14]  Gert Pfurtscheller,et al.  Characterization of four-class motor imagery EEG data for the BCI-competition 2005 , 2005, Journal of neural engineering.

[15]  G Pfurtscheller,et al.  EEG-based communication: improved accuracy by response verification. , 1998, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[16]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[17]  David E Thompson,et al.  Performance assessment in brain-computer interface-based augmentative and alternative communication , 2013, BioMedical Engineering OnLine.

[18]  M. Matteucci,et al.  The Utility Metric: A Novel Method to Assess the Overall Performance of Discrete Brain–Computer Interfaces , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.