Improved inference and autotyping in EEG-based BCI typing systems

The RSVP Keyboard™ is a brain-computer interface (BCI)-based typing system for people with severe physical disabilities, specifically those with locked-in syndrome (LIS). It uses signals from an electroencephalogram (EEG) combined with information from an n-gram language model to select letters to be typed. One characteristic of the system as currently configured is that it does not keep track of past EEG observations, i.e., observations of user intent made while the user was in a different part of a typed message. We present a principled approach for taking all past observations into account, and show that this method results in a 20% increase in simulated typing speed under a variety of conditions on realistic stimuli. We also show that this method allows for a principled and improved estimate of the probability of the backspace symbol, by which mis-typed symbols are corrected. Finally, we demonstrate the utility of automatically typing likely letters in certain contexts, a technique that achieves increased typing speed under our new method, though not under the baseline approach.

[1]  Per Ola Kristensson,et al.  The Imagination of Crowds: Conversational AAC Language Modeling using Crowdsourcing and Large Data Sources , 2011, EMNLP.

[2]  Gregory W. Lesher LEVERAGING WORD PREDICTION TO IMPROVE CHARACTER PREDICTION IN A SCANNING CONFIGURATION , 2002 .

[3]  E. W. Sellers,et al.  Toward enhanced P300 speller performance , 2008, Journal of Neuroscience Methods.

[4]  J. Friedman Regularized Discriminant Analysis , 1989 .

[5]  Brian Roark,et al.  Scanning methods and language modeling for binary switch typing , 2010, SLPAT@NAACL.

[6]  Brian Roark,et al.  The Application of Natural Language Processing to Augmentative and Alternative Communication , 2012, Assistive technology : the official journal of RESNA.

[7]  Brian Roark,et al.  Improved accuracy using recursive Bayesian estimation based language model fusion in ERP-based BCI typing systems , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[8]  B. Blankertz,et al.  (C)overt attention and visual speller design in an ERP-based brain-computer interface , 2010, Behavioral and Brain Functions.

[9]  Brian Roark,et al.  RSVP keyboard: An EEG based typing interface , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[10]  D. Jeffery Higginbotham,et al.  Evaluation of keystroke savings across five assistive communication technologies , 1992 .

[11]  Bob Carpenter,et al.  Scaling High-Order Character Language Models to Gigabytes , 2005, ACL 2005.

[12]  Simon P. Levine,et al.  Effect of a word prediction feature on user performance , 1996 .

[13]  Ian H. Witten,et al.  The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression , 1991, IEEE Trans. Inf. Theory.

[14]  Brian Roark,et al.  Huffman scanning: Using language models within fixed-grid keyboard emulation , 2013, Comput. Speech Lang..