Navigation-synchronized multimodal control wheelchair from brain to alternative assistive technologies for persons with severe disabilities

Currently, electric wheelchairs are commonly used to improve mobility in disabled people. In severe cases, the user is unable to control the wheelchair by themselves because his/her motor functions are disabled. To restore mobility function, a brain-controlled wheelchair (BCW) would be a promising system that would allow the patient to control the wheelchair by their thoughts. P300 is a reliable brain electrical signal, a component of visual event-related potentials (ERPs), that could be used for interpreting user commands. This research aimed to propose a prototype BCW to allowed severe motor disabled patients to practically control a wheelchair for use in their home environment. The users were able to select from 9 possible destination commands in the automatic mode and from 4 directional commands (forward, backward, turn left and right) in the shared-control mode. These commands were selected via the designed P300 processing system. The wheelchair was steered to the desired location by the implemented navigation system. Safety of the user was ensured during wheelchair navigation due to the included obstacle detection and avoidance features. A combination of P300 and EOG was used as a hybrid BCW system. The user could fully operate the system such as enabling P300 detection system, mode shifting and stop/cancelation command by performing a different consecutive blinks to generate eye blinking patterns. The results revealed that the prototype BCW could be operated in either of the proposed modes. With the new design of the LED-based P300 stimulator, the average accuracies of the P300 detection algorithm in the shared-control and automatic modes were 95.31 and 83.42% with 3.09 and 3.79 bits/min, respectively. The P300 classification error was acceptable, as the user could cancel an incorrect command by blinking 2 times. Moreover, the proposed navigation system had a flexible design that could be interfaced with other assistive technologies. This research developed 3 alternative input modules: an eye tracker module and chin and hand controller modules. The user could select the most suitable assistive technology based on his/her level of disability. Other existing assistive technologies could also be connected to the proposed system in the future using the same protocol.

[1]  Hugh Durrant-Whyte,et al.  Simultaneous localization and mapping (SLAM): part II , 2006 .

[2]  Iñaki Iturrate,et al.  A Noninvasive Brain-Actuated Wheelchair Based on a P300 Neurophysiological Protocol and Automated Navigation , 2009, IEEE Transactions on Robotics.

[3]  Hugh Herr,et al.  Exoskeletons and orthoses: classification, design challenges and future directions , 2009, Journal of NeuroEngineering and Rehabilitation.

[4]  M. Ghovanloo,et al.  The Tongue Enables Computer and Wheelchair Control for People with Spinal Cord Injury , 2013, Science Translational Medicine.

[5]  Takashi Watanabe,et al.  Gyro-Mouse for the Disabled: 'Click' and 'Position' Control of the Mouse Cursor , 2007 .

[6]  Noam Sobel,et al.  Sniffing enables communication and environmental control for the severely disabled , 2010, Proceedings of the National Academy of Sciences.

[7]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[8]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[9]  Álvaro Fernández-Rodríguez,et al.  Wheelchair navigation with an audio-cued, two-class motor imagery-based brain-computer interface system , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[10]  Kohei Arai,et al.  Eyes Based Eletric Wheel Chair Control System - I (eye) can control Electric Wheel Chair - , 2011 .

[11]  S. Galetta,et al.  The central visual field in homonymous hemianopia. , 1997 .

[12]  Touradj Ebrahimi,et al.  An efficient P300-based brain–computer interface for disabled subjects , 2008, Journal of Neuroscience Methods.

[13]  Peng Xu,et al.  Shared control of a brain-actuated intelligent wheelchair , 2014, Proceeding of the 11th World Congress on Intelligent Control and Automation.

[14]  Marta Kutas,et al.  Parafoveal perception during sentence reading? An ERP paradigm using rapid serial visual presentation (RSVP) with flankers. , 2011, Psychophysiology.

[15]  Yuanqing Li,et al.  Control of a Wheelchair in an Indoor Environment Based on a Brain–Computer Interface and Automated Navigation , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[16]  Danny Wee-Kiat Ng,et al.  Development of an Autonomous BCI Wheelchair , 2014, 2014 IEEE Symposium on Computational Intelligence in Brain Computer Interfaces (CIBCI).

[17]  Yodchanan Wongsawat,et al.  Minimal-assisted SSVEP-based brain-computer interface device , 2012, Proceedings of The 2012 Asia Pacific Signal and Information Processing Association Annual Summit and Conference.

[18]  Michael Jones,et al.  A Sip-and-Puff Wireless Remote Control for the Apple iPod , 2008, Assistive technology : the official journal of RESNA.

[19]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[20]  Jayme Knutson,et al.  Neuromuscular Electrical Stimulation for Motor Restoration in Hemiplegia , 2008, Topics in stroke rehabilitation.

[21]  Christian Mandel,et al.  Navigating a smart wheelchair with a brain-computer interface interpreting steady-state visual evoked potentials , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Haihong Zhang,et al.  A Brain-Controlled Wheelchair Based on P300 and Path Guidance , 2006, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[23]  G. Vanacker,et al.  Adaptive Shared Control of a Brain-Actuated Simulated Wheelchair , 2007, 2007 IEEE 10th International Conference on Rehabilitation Robotics.

[24]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.

[25]  D. Venkataraman,et al.  Eye movement based electronic wheel chair for physically challenged persons , 2014 .

[26]  Arthur Prochazka,et al.  Tooth-Click Control of a Hands-Free Computer Interface , 2008, IEEE Transactions on Biomedical Engineering.

[27]  Yodchanan Wongsawat,et al.  Hybrid EEG-EOG brain-computer interface system for practical machine control , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[28]  Yodchanan Wongsawat,et al.  Automated navigation system for eye-based wheelchair controls , 2014, The 7th 2014 Biomedical Engineering International Conference.

[29]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[30]  Oskar von Stryk,et al.  Hector Open Source Modules for Autonomous Mapping and Navigation with Rescue Robots , 2013, RoboCup.

[31]  Johan A. K. Suykens,et al.  Feature Extraction and Classification of EEG Signals for Rapid P300 Mind Spelling , 2009, 2009 International Conference on Machine Learning and Applications.

[32]  Xiao Li,et al.  The vocal joystick:: evaluation of voice-based cursor control techniques , 2006, Assets '06.

[33]  K. Dobosz,et al.  Brain-computer interface for mobile devices , 2015 .

[34]  S. Galetta,et al.  The central visual field in homonymous hemianopia. Evidence for unilateral foveal representation. , 1997, Archives of neurology.

[35]  Brice Rebsamen,et al.  A brain controlled wheelchair to navigate in familiar environments. , 2010, IEEE transactions on neural systems and rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[36]  Giulio Fontana,et al.  Brain Control of a Smart Wheelchair , 2008 .

[37]  Kyuwan Choi,et al.  Control of a Wheelchair by Motor Imagery in Real Time , 2008, IDEAL.

[38]  Y. Wongsawat,et al.  Illuminant effect on LCD and LED stimulators for P300-based brain-controlled wheelchair , 2012, The 4th 2011 Biomedical Engineering International Conference.

[39]  Prabin Kumar Padhy,et al.  Feature Extraction and Classification of EEG Signal , 2011 .

[40]  Kazuo Tanaka,et al.  Electroencephalogram-based control of an electric wheelchair , 2005, IEEE Transactions on Robotics.

[41]  Y. Wongsawat,et al.  Semi-automatic P300-based brain-controlled wheelchair , 2012, 2012 ICME International Conference on Complex Medical Engineering (CME).