RSVP IconMessenger: icon-based brain-interfaced alternative and augmentative communication

One of the principal application areas for brain-computer interface (BCI) technology is augmentative and alternative communication (AAC), typically used by people with severe speech and physical disabilities (SSPI). Existing word- and phrase-based AAC solutions that employ BCIs that utilize electroencephalography (EEG) are sometimes supplemented by icons. Icon-based BCI systems that use binary signaling methods, such as P300 detection, combine hierarchical layouts with some form of scanning. The rapid serial visual presentation (RSVP) IconMessenger combines P300 signal detection with the icon-based semantic message construction system of iconCHAT. Language models are incorporated in the inference engine and some modifications that facilitate the use of RSVP were performed such as icon semantic role order selection and the tight fusion of language evidence and EEG evidence. The results of a study conducted with 10 healthy participants suggest that the system has potential as an AAC system in real-time typi...

[1]  Touradj Ebrahimi,et al.  An efficient P300-based brain–computer interface for disabled subjects , 2008, Journal of Neuroscience Methods.

[2]  Rupal Patel,et al.  Non-Syntactic Word Prediction for AAC , 2012, SLPAT@HLT-NAACL.

[3]  P. J. Green,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[4]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[5]  Brian Roark,et al.  Offline analysis of context contribution to ERP-based typing BCI performance. , 2013, Journal of neural engineering.

[6]  Andrew Jinks,et al.  Consumer response to AAC devices: Acquisition, training, use, and satisfaction , 1994 .

[7]  Brian Roark,et al.  RSVP keyboard: An EEG based typing interface , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[8]  A. Kübler,et al.  Flashing characters with famous faces improves ERP-based brain–computer interface performance , 2011, Journal of neural engineering.

[9]  Howard Bowman,et al.  The cost of space independence in P300-BCI spellers , 2013, Journal of NeuroEngineering and Rehabilitation.

[10]  Norman Alm,et al.  Computer-aided conversation: A prototype system for nonspeaking people with physical disabilities , 1994, Applied Psycholinguistics.

[11]  Tonio Wandmacher,et al.  Training Language Models without Appropriate Language Resources: Experiments with an AAC System for Disabled People , 2006, LREC.

[12]  Per Ola Kristensson,et al.  The Imagination of Crowds: Conversational AAC Language Modeling using Crowdsourcing and Large Data Sources , 2011, EMNLP.

[13]  E. W. Sellers,et al.  Toward enhanced P300 speller performance , 2008, Journal of Neuroscience Methods.

[14]  Brian Roark,et al.  Improved accuracy using recursive Bayesian estimation based language model fusion in ERP-based BCI typing systems , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[15]  Martin F. Porter,et al.  An algorithm for suffix stripping , 1997, Program.

[16]  Yang Liu,et al.  A novel task-oriented optimal design for P300-based brain–computer interfaces , 2014, Journal of neural engineering.

[17]  F Babiloni,et al.  A comparison of classification techniques for a gaze-independent P300-based brain-computer interface. , 2012, Journal of neural engineering.

[18]  M S Treder,et al.  Gaze-independent brain–computer interfaces based on covert attention and feature attention , 2011, Journal of neural engineering.

[19]  I. Scott MacKenzie,et al.  SAK: Scanning ambiguous keyboard for efficient one-key text entry , 2010, TCHI.

[20]  Laura J. Ball,et al.  Acceptance of Augmentative and Alternative Communication Technology by Persons with Amyotrophic Lateral Sclerosis , 2004 .

[21]  D. Hu,et al.  Gaze independent brain–computer speller with covert visual search tasks , 2011, Clinical Neurophysiology.

[22]  Kathleen F. McCoy,et al.  Corpus studies in word prediction , 2007, Assets '07.

[23]  E. John,et al.  Evoked-Potential Correlates of Stimulus Uncertainty , 1965, Science.

[24]  Jessica D. Bayliss,et al.  Changing the P300 Brain Computer Interface , 2004, Cyberpsychology Behav. Soc. Netw..

[25]  J. Wolpaw,et al.  A P300-based brain–computer interface for people with amyotrophic lateral sclerosis , 2008, Clinical Neurophysiology.

[26]  J. Friedman.  Regularized Discriminant Analysis , 1989 .

[27]  D. Beukelman,et al.  Augmentative & Alternative Communication: Supporting Children & Adults With Complex Communication Needs , 2006 .

[28]  C. A. Dairaghi,et al.  Concurrent neuromechanical and functional gains following upper-extremity power training post-stroke , 2013, Journal of NeuroEngineering and Rehabilitation.

[29]  Benjamin Blankertz,et al.  A novel brain-computer interface based on the rapid serial visual presentation paradigm , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[30]  C. Fillmore.  FRAME SEMANTICS AND THE NATURE OF LANGUAGE * , 1976 .