Quadcopter flight control using a low-cost hybrid interface with EEG-based classification and eye tracking

We propose a wearable hybrid interface where eye movements and mental concentration directly influence the control of a quadcopter in three-dimensional space. This noninvasive and low-cost interface addresses limitations of previous work by supporting users to complete their complicated tasks in a constrained environment in which only visual feedback is provided. The combination of the two inputs augments the number of control commands to enable the flying robot to travel in eight different directions within the physical environment. Five human subjects participated in the experiments to test the feasibility of the hybrid interface. A front view camera on the hull of the quadcopter provided the only visual feedback to each remote subject on a laptop display. Based on the visual feedback, the subjects used the interface to navigate along pre-set target locations in the air. The flight performance was evaluated by comparing with a keyboard-based interface. We demonstrate the applicability of the hybrid interface to explore and interact with a three-dimensional physical space through a flying robot.

[1]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[2]  G. Pfurtscheller,et al.  How many people are able to operate an EEG-based brain-computer interface (BCI)? , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[3]  Dongheng Li,et al.  openEyes: a low-cost head-mounted eye-tracking solution , 2006, ETRA.

[4]  Sungho Jo,et al.  A Low-Cost EEG System-Based Hybrid Brain-Computer Interface for Humanoid Robot Navigation and Recognition , 2013, PloS one.

[5]  Rabab K Ward,et al.  A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals , 2007, Journal of neural engineering.

[6]  G R Müller-Putz,et al.  Toward smarter BCIs: extending BCIs through hybridization and intelligent control , 2012, Journal of neural engineering.

[7]  David Fitzpatrick,et al.  Neuroscience, 3rd ed. , 2004 .

[8]  Laura Chamberlain.  Eye Tracking Methodology; Theory and Practice , 2007 .

[9]  Thorsten O. Zander,et al.  Combining Eye Gaze Input With a Brain–Computer Interface for Touchless Human–Computer Interaction , 2010, Int. J. Hum. Comput. Interact..

[10]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[11]  Shumin Zhai,et al.  Keeping an eye for HCI , 1999, XII Brazilian Symposium on Computer Graphics and Image Processing (Cat. No.PR00481).

[12]  Jaeseung Jeong,et al.  Toward Brain-Actuated Humanoid Robots: Asynchronous Direct Control Using an EEG-Based BCI , 2012, IEEE Transactions on Robotics.

[13]  S. Schultz.  Principles of Neural Science, 4th ed. , 2001 .

[14]  Han-Jeong Hwang,et al.  Neurofeedback-based motor imagery training for brain–computer interface (BCI) , 2009, Journal of Neuroscience Methods.

[15]  Federico Manuri,et al.  A Kinect-based natural interface for quadrotor control , 2011, Entertain. Comput..

[16]  F Cincotti,et al.  Current trends in hardware and software for brain–computer interfaces (BCIs) , 2011, Journal of neural engineering.

[17]  G. Pfurtscheller,et al.  Enhancement of left-right sensorimotor EEG differences during feedback-regulated motor imagery. , 1999, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[18]  David J. Hand,et al.  A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems , 2001, Machine Learning.

[19]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[20]  A. Doud,et al.  Continuous Three-Dimensional Control of a Virtual Helicopter Using a Motor Imagery Based Brain-Computer Interface , 2011, PloS one.

[21]  Jaeseung Jeong,et al.  Brain-actuated humanoid robot navigation control using asynchronous Brain-Computer Interface , 2011, 2011 5th International IEEE/EMBS Conference on Neural Engineering.

[22]  W W Abbott,et al.  Ultra-low-cost 3D gaze estimation: an intuitive high information throughput compliment to direct brain–machine interfaces , 2012, Journal of neural engineering.

[23]  Dennis J. McFarland,et al.  Brain-computer interface (BCI) operation: signal and noise during early training sessions , 2005, Clinical Neurophysiology.

[24]  Tommy Strandvall,et al.  Eye Tracking in Human-Computer Interaction and Usability Research , 2009, INTERACT.

[25]  Riccardo Poli,et al.  Towards cooperative brain-computer interfaces for space navigation , 2013, IUI '13.

[26]  K. Lafleur,et al.  Quadcopter control in three-dimensional space using a noninvasive motor imagery-based brain–computer interface , 2013, Journal of neural engineering.

[27]  Reinhold Scherer,et al.  Steady-state visual evoked potential (SSVEP)-based communication: impact of harmonic frequency components , 2005, Journal of neural engineering.

[28]  Bin He,et al.  EEG Control of a Virtual Helicopter in 3-Dimensional Space Using Intelligent Control Strategies , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[29]  Shaohan Hu,et al.  NeuroPhone: brain-mobile phone interface using a wireless EEG headset , 2010, MobiHeld '10.

[30]  Kee-Eung Kim,et al.  A POMDP approach to P300-based brain-computer interfaces , 2010, IUI '10.

[31]  Brendan Z. Allison,et al.  Brain-Computer Interfaces: Revolutionizing Human-Computer Interaction , 2013 .

[32]  A. Frolov,et al.  Brain-Computer Interface Based on Generation of Visual Images , 2011, PloS one.

[33]  Minho Kim,et al.  Hybrid EEG and eye movement interface to multi-directional target selection , 2013, 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[34]  D. Borghetti,et al.  A low-cost interface for control of computer functions by means of eye movements , 2007, Comput. Biol. Medicine.