Flight simulation using a Brain-Computer Interface: A pilot, pilot study

&NA; As Brain‐Computer Interface (BCI) systems advance for uses such as robotic arm control it is postulated that the control paradigms could apply to other scenarios, such as control of video games, wheelchair movement or even flight. The purpose of this pilot study was to determine whether our BCI system, which involves decoding the signals of two 96‐microelectrode arrays implanted into the motor cortex of a subject, could also be used to control an aircraft in a flight simulator environment. The study involved six sessions in which various parameters were modified in order to achieve the best flight control, including plane type, view, control paradigm, gains, and limits. Successful flight was determined qualitatively by evaluating the subject's ability to perform requested maneuvers, maintain flight paths, and avoid control losses such as dives, spins and crashes. By the end of the study, it was found that the subject could successfully control an aircraft. The subject could use both the jet and propeller plane with different views, adopting an intuitive control paradigm. From the subject's perspective, this was one of the most exciting and entertaining experiments she had performed in two years of research. In conclusion, this study provides a proof‐of‐concept that traditional motor cortex signals combined with a decoding paradigm can be used to control systems besides a robotic arm for which the decoder was developed. Aside from possible functional benefits, it also shows the potential for a new recreational activity for individuals with disabilities who are able to master BCI control. HighlightsA Brain‐Computer Interface controlled flight simulator is tested in a pilot study.The subject successfully controlled aircrafts using signals from her motor cortex.She learned to control the system with different planes, views, and locations.This system could potentially be used for transport control or recreation.

[1]  Faisal Karmali,et al.  Environmental control by a brain-computer interface , 2000, Proceedings of the 22nd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (Cat. No.00CH37143).

[2]  Michael W. Haas,et al.  Postural Instability and Motion Sickness in a Fixed-Base Flight Simulator , 2000, Hum. Factors.

[3]  Ad Aertsen,et al.  Invasive brain–machine interfaces: a survey of paralyzed patients’ attitudes, knowledge and methods of information retrieval , 2015, Journal of neural engineering.

[4]  R A Normann,et al.  The Utah intracortical Electrode Array: a recording structure for potential brain-computer interfaces. , 1997, Electroencephalography and clinical neurophysiology.

[5]  Stuart D. Harshbarger,et al.  An Overview of the Developmental Process for the Modular Prosthetic Limb , 2011 .

[6]  G Calhoun,et al.  Brain-computer interfaces based on the steady-state visual-evoked response. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[7]  M L Boninger,et al.  Ten-dimensional anthropomorphic arm control in a human brain−machine interface: difficulties, solutions, and limitations , 2015, Journal of neural engineering.

[8]  Erik Scheme,et al.  A real-time virtual integration environment for the design and development of neural prosthetic systems , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[9]  A. Doud,et al.  Continuous Three-Dimensional Control of a Virtual Helicopter Using a Motor Imagery Based Brain-Computer Interface , 2011, PloS one.

[10]  Wei Wang,et al.  Collaborative Approach in the Development of High‐Performance Brain–Computer Interfaces for a Neuroprosthetic Arm: Translation from Animal Models to Human Control , 2014, Clinical and translational science.

[11]  Lihua Huang,et al.  On the Control of the Berkeley Lower Extremity Exoskeleton (BLEEX) , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[12]  Gernot R. Müller-Putz,et al.  Self-Paced (Asynchronous) BCI Control of a Wheelchair in Virtual Environments: A Case Study with a Tetraplegic , 2007, Comput. Intell. Neurosci..

[13]  Vikash Gilja,et al.  Assessment of brain–machine interfaces from the perspective of people with paralysis , 2015, Journal of neural engineering.

[14]  Minho Kim,et al.  Quadcopter flight control using a low-cost hybrid interface with EEG-based classification and eye tracking , 2014, Comput. Biol. Medicine.

[15]  Andrew B Schwartz,et al.  Neuroprosthetic control and tetraplegia – Authors'reply , 2013, The Lancet.

[16]  A. P. Georgopoulos,et al.  Neuronal population coding of movement direction. , 1986, Science.

[17]  Michael L Boninger,et al.  Functional priorities, assistive technology, and brain-computer interfaces after spinal cord injury. , 2013, Journal of rehabilitation research and development.

[18]  J. Huggins,et al.  What would brain-computer interface users want: opinions and priorities of potential users with spinal cord injury. , 2015, Archives of physical medicine and rehabilitation.

[19]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[20]  Samuel T. Clanton,et al.  Brain-Computer Interface Control of an Anthropomorphic Robotic Arm , 2011 .

[21]  A. Schwartz,et al.  High-performance neuroprosthetic control by an individual with tetraplegia , 2013, The Lancet.

[22]  José del R. Millán,et al.  BNCI Horizon 2020: Towards a Roadmap for the BCI Community , 2015 .

[23]  Touradj Ebrahimi,et al.  Brain-computer interface in multimedia communication , 2003, IEEE Signal Process. Mag..

[24]  Kapil D. Katyal,et al.  Revolutionizing Prosthetics—Phase 3 , 2013 .

[25]  M.M. Moore,et al.  Real-world applications for brain-computer interface technology , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[26]  K. Anderson Targeting recovery: priorities of the spinal cord-injured population. , 2004, Journal of neurotrauma.

[27]  Michael P McLoughlin DARPA Revolutionizing Prosthetics 2009 , 2009 .

[28]  Klaus-Robert Müller,et al.  The Berlin Brain-Computer Interface (BBCI) – towards a new communication channel for online control in gaming applications , 2007, Multimedia Tools and Applications.

[29]  Sneh Anand,et al.  Brain Computer Interface Based Real Time Control of Wheelchair Using Electroencephalogram , 2011 .

[30]  Tobias Kaufmann,et al.  Long-term independent brain-computer interface home use improves quality of life of a patient in the locked-in state: a case study. , 2015, Archives of physical medicine and rehabilitation.