Integrated Human-Machine Interaction System: ERP-SSVEP and Eye Tracking Based Technologies

Human-Machine Interaction (HMI) requires a multidisciplinary research study mainly focused on interaction modalities between humans and machines. In this paper, we introduced an integrated HMI system using electrical brainwave signal (or called Electroencephalography: EEG) and eye tracking of pupil movement (or called ET), whose are used as an alternative channel to communicate with others for people with disabilities. In this experiment, the target and non-target visual stimuli of EEG-based HMI system on the basis of event-related potential (ERP) and steady state visually evoked potential (SSVEP) signals have been performed. For the ET based framework, we proposed the user-friendly virtual keyboard typing for Thai language, i.e., free-form and automatic typing modes. The results showed that the integrated HMI using ERP-SSVEP yielded an average accuracy of 97.4% and reaction time approximately was 724.2 millisecond for control commands. The automatic typing mode performed an average accuracy of 97%, with an average printing time of 6.17 seconds per word for ET based virtual Thai keyboard.

[1]  Gernot R. Müller-Putz,et al.  "Virtual keyboard" controlled by spontaneous EEG activity , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Linden J. Ball,et al.  Eye Tracking in Human-Computer Interaction and Usability Research : Current Status and Future Prospects , 2004 .

[3]  H. Flor,et al.  A spelling device for the paralysed , 1999, Nature.

[4]  Hiran Ekanayake,et al.  P300 and Emotiv EPOC: Does Emotiv EPOC capture real EEG? , 2010 .

[5]  Hubert Cecotti.  Effect of the Stimulus Duty Cycle on Steady-State Visual Evoked Potential detection , 2010 .

[6]  Montri Phothisonothai,et al.  Effects of visual flickering stimuli for event-related potential recording , 2016, 2016 8th International Conference on Knowledge and Smart Technology (KST).

[7]  Montri Phothisonothai,et al.  Design of User-Friendly Virtual Thai Keyboard Based on Eye-Tracking Controlled System , 2018, 2018 18th International Symposium on Communications and Information Technologies (ISCIT).

[8]  David Rozado,et al.  Improving the Performance of an EEG-Based Motor Imagery Brain Computer Interface Using Task Evoked Changes in Pupil Diameter , 2015, PloS one.

[9]  J. Wolpaw,et al.  A P300 event-related potential brain–computer interface (BCI): The effects of matrix size and inter stimulus interval on performance , 2006, Biological Psychology.

[10]  C. Herrmann.  Human EEG responses to 1–100 Hz flicker: resonance phenomena in visual cortex and their potential correlation to cognitive phenomena , 2001, Experimental Brain Research.

[11]  Sebastian von Mammen,et al.  Towards EEG-based eye-tracking for interaction design in head-mounted devices , 2017, 2017 IEEE 7th International Conference on Consumer Electronics - Berlin (ICCE-Berlin).

[12]  Chang-Hwan Im,et al.  Development of an SSVEP-based BCI spelling system adopting a QWERTY-style LED keyboard , 2012, Journal of Neuroscience Methods.

[13]  Rajesh Singla,et al.  Virtual keyboard BCI using Eye blinks in EEG , 2010, 2010 IEEE 6th International Conference on Wireless and Mobile Computing, Networking and Communications.

[14]  D. McFarland,et al.  An auditory brain–computer interface (BCI) , 2008, Journal of Neuroscience Methods.

[15]  M. Phothisonothai,et al.  Time-frequency based coherence analysis of red and green flickering visual stimuli for EEG-controlled applications , 2017, 2017 9th International Conference on Knowledge and Smart Technology (KST).

[16]  Hubert Cecotti,et al.  A Multimodal Gaze-Controlled Virtual Keyboard , 2016, IEEE Transactions on Human-Machine Systems.

[17]  Shumin Zhai,et al.  Chinese input with keyboard and eye-tracking: an anatomical study , 2001, CHI.

[18]  Montri Phothisonothai,et al.  Time-frequency analysis of duty cycle changing on steady-state visual evoked potential: EEG recording , 2014, Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific.

[19]  Ching-I Wu.  HCI and Eye Tracking Technology for Learning Effect , 2012 .

[20]  G. Sperling,et al.  Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency. , 2006, Cerebral cortex.