The Mind-Writing Pupil: A Human-Computer Interface Based on Decoding of Covert Attention through Pupillometry

We present a new human-computer interface that is based on decoding of attention through pupillometry. Our method builds on the recent finding that covert visual attention affects the pupillary light response: Your pupil constricts when you covertly (without looking at it) attend to a bright, compared to a dark, stimulus. In our method, participants covertly attend to one of several letters with oscillating brightness. Pupil size reflects the brightness of the selected letter, which allows us–with high accuracy and in real time–to determine which letter the participant intends to select. The performance of our method is comparable to the best covert-attention brain-computer interfaces to date, and has several advantages: no movement other than pupil-size change is required; no physical contact is required (i.e. no electrodes); it is easy to use; and it is reliable. Potential applications include: communication with totally locked-in patients, training of sustained attention, and ultra-secure password input.

[1]  N. Birbaumer Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. , 2006, Psychophysiology.

[2]  Benjamin Blankertz,et al.  THE BERLIN BRAIN-COMPUTER INTERFACE PRESENTS THE NOVEL MENTAL TYPEWRITER HEX-O-SPELL , 2006 .

[3]  Anish A. Sarma,et al.  Clinical translation of a high-performance neural prosthesis , 2015, Nature Medicine.

[4]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[5]  Benjamin Blankertz,et al.  Gaze-independent BCI-spelling using rapid serial visual presentation (RSVP) , 2013, Clinical Neurophysiology.

[6]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[7]  Per B. Brockhoff,et al.  lmerTest Package: Tests in Linear Mixed Effects Models , 2017 .

[8]  Sebastiaan Mathôt,et al.  PyGaze: An open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments , 2014, Behavior research methods.

[9]  B. Blankertz,et al.  (C)overt attention and visual speller design in an ERP-based brain-computer interface , 2010, Behavioral and Brain Functions.

[10]  John P. Donoghue,et al.  Bridging the Brain to the World: A Perspective on Neural Interface Systems , 2008, Neuron.

[11]  Jonathan Grainger,et al.  The pupillary light response reflects exogenous attention and inhibition of return. , 2014, Journal of vision.

[12]  J. Wolpaw,et al.  Does the ‘P300’ speller depend on eye gaze? , 2010, Journal of neural engineering.

[13]  S. Coyle,et al.  Brain–computer interfaces: a review , 2003 .

[14]  F. Cincotti,et al.  Eye-gaze independent EEG-based brain–computer interfaces for communication , 2012, Journal of neural engineering.

[15]  Manabu Iguchi,et al.  Entry of inclined hydrophobic and hydrophilic circular cylinders into water , 2011, J. Vis..

[16]  M S Treder,et al.  Gaze-independent brain–computer interfaces based on covert attention and feature attention , 2011, Journal of neural engineering.

[17]  J. Grainger,et al.  The Pupillary Light Response Reveals the Focus of Covert Visual Attention , 2013, PloS one.

[18]  Christof Koch,et al.  Pupil responses allow communication in locked-in syndrome patients , 2013, Current Biology.

[19]  B. Laeng,et al.  The Eye Pupil Adjusts to Imaginary Light , 2014, Psychological science.

[20]  Sebastiaan Mathôt,et al.  New Light on the Mind’s Eye , 2015, Current directions in psychological science.

[21]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[22]  D. van Norren,et al.  Visual acuity measured with pupil responses to checkerboard stimuli. , 1980, Investigative ophthalmology & visual science.

[23]  Brendan Z. Allison,et al.  P300 brain computer interface: current challenges and emerging trends , 2012, Front. Neuroeng..

[24]  Denis Cousineau,et al.  Confidence intervals in within-subject designs: A simpler solution to Loftus and Masson's method , 2005 .

[25]  K. Ukai,et al.  Spatial pattern as a stimulus to the pupillary system. , 1985, Journal of the Optical Society of America. A, Optics and image science.

[26]  George A. Alvarez,et al.  Tracking the allocation of attention using human pupillary oscillations , 2013, Front. Psychol..

[27]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[28]  Ralf Engbert,et al.  Microsaccades uncover the orientation of covert attention , 2003, Vision Research.

[29]  Latency of pupillary reflex to light stimulation and its relationship to aging. , 1965 .

[30]  Jan Theeuwes,et al.  OpenSesame: An open-source, graphical experiment builder for the social sciences , 2011, Behavior Research Methods.

[31]  S. Sirois,et al.  Pupillometry , 2012, Perspectives on psychological science : a journal of the Association for Psychological Science.

[32]  D. Bates,et al.  Linear Mixed-Effects Models using 'Eigen' and S4 , 2015 .

[33]  E. Castet,et al.  Intrasaccadic perception triggers pupillary constriction , 2015, PeerJ.

[34]  Paola Binda,et al.  Attention to Bright Surfaces Enhances the Pupillary Light Reflex , 2013, The Journal of Neuroscience.

[35]  Thierry Dutoit,et al.  Performance of the Emotiv Epoc headset for P300-based applications , 2013, Biomedical engineering online.

[36]  Edwin S. Dalmaijer,et al.  Is the low-cost EyeTribe eye tracker any good for research? , 2014 .

[37]  Glenn R. Gibson,et al.  Klebsiella pneumoniae subsp. pneumoniae–bacteriophage combination from the caecal effluent of a healthy woman , 2015, PeerJ.

[38]  F. Gerstenbrand,et al.  Varieties of the locked-in syndrome , 1979, Journal of Neurology.

[39]  I. Scott MacKenzie,et al.  Effects of feedback and dwell time on eye typing speed and accuracy , 2006, Universal Access in the Information Society.

[40]  Luciano Gamberini,et al.  Covert Visuospatial Attention Orienting in a Brain-Computer Interface for Amyotrophic Lateral Sclerosis Patients , 2013, Neurorehabilitation and neural repair.

[41]  Elaine Astrand,et al.  Selective visual attention to drive cognitive brain–machine interfaces: from concepts to neurofeedback and rehabilitation applications , 2014, Front. Syst. Neurosci..

[42]  Peng Yuan,et al.  A study of the existing problems of estimating the information transfer rate in online brain–computer interfaces , 2013, Journal of neural engineering.

[43]  Dean J Krusienski,et al.  A comparison of classification techniques for the P300 Speller , 2006, Journal of neural engineering.