Embedded brain reading

Current autonomous robots and interfaces are far from exhibiting the adaptability of biological beings regarding changes in their environment or during interaction. They are not always able to provide humans the best and a situation-specific support. Giving the robot or its interface insight into the human mind can open up new possibilities for the integration of human cognitive resources into robots and interfaces, i.e., into their intelligent control systems, and can particularly improve humanmachine interaction. In this thesis embedded Brain Reading (eBR) is developed. It empowers a human-machine interface (HMI), which can be a robotic system, to infer the human’s intention and hence her/his upcoming interaction behavior based on the context of the interaction and the human’s brain state. To enable eBR, an automatic context recognition or generation as well as online, single-trial brain signal decoding, i.e., Brain Reading (BR) for the detection of specific brain states, are required. The human’s electroencephalogram (EEG) recorded from the head’s surface is used in this work as a measure of brain activity. Experiments are conducted in controlled experimental setups, where subjects have to execute differently complex and demanding simple and dual-task behavior as it is performed during human-machine interaction. Using these experiments the applicability and reliability of BR is confirmed as well as training procedures for BR are improved. Furthermore, a formal model for eBR is developed and shown to be applicable for different implementations of eBR. The formal model is the first step to check implementations of eBR for their correctness and completeness. By means of robotic applications for tele-manipulation and rehabilitation it is further shown that eBR can be applied to either adapt or to drive HMIs, i.e., can be used to implement predictive HMIs for passive or active support. In case that eBR is applied for passive support, it is shown that malfunction of the whole system can be avoided. On the other hand, in case that eBR is applied for active support, i.e., to actively drive an HMI, it is shown that an individual adaptation of the support with respect to the requirements of different users can be facilitated by utilizing multi-modal signal analysis in eBR. Finally, it is shown that even in case of passive support eBR can measurably improve human-machine interaction. Zusammenfassung Autonome Roboter und Mensch-Maschine Schnittstellen sind heutzutage noch nicht so adaptiv und flexibel im Bezug auf Veränderungen in ihrer Umgebung oder sich ändernden Anforderungen ihres Interaktionspartners, wie es biologische Systeme sind. Aus diesem Grund erfüllen solche technischen Systeme nur eingeschränkt die Anforderung, Menschen situationsspezifisch und entsprechend wechselnden Gegebenheiten optimal zu unterstützen. Um dies zu ändern, ist es notwendig, dass robotische Systeme und ihre Schnittstellen Einsicht in die Gedankenwelt des Menschen erhalten. Dies ermöglicht es dem technischen System, menschliche kognitive Ressourcen zur Optimierung ihrer intelligenten Kontrolle und somit zur Optimierung der Interaktion zwischen Mensch und Maschine zu nutzen. In dieser Arbeit wird embedded Brain Reading (eBR) entwickelt. Es befähigt eine Mensch-Maschine Schnittstelle, die ein robotisches System sein kann, Annahmen über die Absichten des interagierenden Menschen aufzustellen und damit zukünftiges Verhalten im Kontext der Interaktion und basierend auf dem ermittelten Zustand des Gehirns vorherzusagen. Dementsprechend wird für die Realisierung von eBR eine automatische Erkennung des Kontextes der Interaktion als auch eine "online" fähige Entschlüsselung von Gehirnaktivität im sogenannten "single-trial" Verfahren, also die Erkennung spezifischer Gehirnzustände mittels Brain Reading (BR), benötigt. Das menschliche, von der Kopfoberfläche gemessene Elektroenzephalogramm (EEG) wird in dieser Arbeit als Methode zur Messung der Gehirnaktivität genutzt. Experimente, in denen Probanden unterschiedlich komplexe und anspruchsvolle Verhalten, so wie sie auch bei der Interaktion zwischen Mensch und Maschine auftreten würden, ausführen müssen, werden in kontrollierten Versuchsumgebungen durchgeführt. Anhand dieser Experimente werden die Zuverlässigkeit von BR gezeigt und Trainingsverfahren für BR verbessert. Des weiteren wird in dieser Arbeit ein formales Model für eBR entwickelt. Für dieses wird gezeigt, dass es für verschiedene Implementierungen von eBR anwendbar ist. Das formale Modell erlaubt Implementierungen von eBR zu verbessern und auf ihre Korrektheit und Vollständigkeit zu überprüfen. Basierend auf Anwendungen aus der Robotik, genauer auf Basis von Telemanipulationsund Rehabilitationsanwendungen, wird außerdem gezeigt, dass eBR genutzt werden kann, um Mensch-Maschine Schnittstellen entweder in ihrer Funktionalität anzupassen, also auf die Anforderungen des Menschen zu optimieren und passiv zu unterstützen oder um die Schnittstelle selbst aktiv zu steuern. Die durch eBR adaptierte oder gesteuerte Schnittstellen werden predictive HMIs genannt. Für den Fall, dass sie für die passive Unterstützung eingesetzt werden, wird gezeigt, dass Fehlfunktionen des Gesamtsystems, welche durch Fehlinterpretationen der Gehirnaktivität potentiell möglich sind, vermieden werden können. Andererseits wird gezeigt, dass der Einsatz von eBR für die aktive Kontrolle von solchen predictive HMIs eine individuelle Anpassung dieser an die Anforderungen des Nutzers oder der Situation, wie z.B. dem Stand der Therapie, ermöglicht. Schlussendlich wird mittels eines Experimentes gezeigt, dass auch für den Fall, dass eBR für die passive Unterstützung eingesetzt wird, die Interaktion zwischen Mensch und Maschine messbar verbessert wird.

[1]  P. Sajda,et al.  Cortically coupled computer vision for rapid image search , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Atsuo Takanishi,et al.  From lamprey to Humanoid: the Design and Control of a Flexible Spine Belly Dancing Humanoid Robot with Inspiration from Biology , 2005, Int. J. Humanoid Robotics.

[3]  G. Rees,et al.  Predicting the Stream of Consciousness from Activity in Human Visual Cortex , 2005, Current Biology.

[4]  Ryan J. Prenger,et al.  Bayesian Reconstruction of Natural Images from Human Brain Activity , 2009, Neuron.

[5]  Frank Kirchner,et al.  On transferring spatial filters in a brain reading scenario , 2011, 2011 IEEE Statistical Signal Processing Workshop (SSP).

[6]  S. Makeig,et al.  Lapses in alertness: coherence of fluctuations in performance and EEG spectrum. , 1993, Electroencephalography and clinical neurophysiology.

[7]  H. Ishiguro,et al.  The thing that should not be: predictive coding and the uncanny valley in perceiving human and humanoid robot actions , 2011, Social cognitive and affective neuroscience.

[8]  Hendrik Wöhrle,et al.  Runtime Calibration of Online EEG based Movement Prediction using EMG Signals , 2014, BIOSIGNALS.

[9]  Mark Peplow,et al.  Mental ping-pong could aid paraplegics , 2004 .

[10]  Elsa Andrea Kirchner,et al.  Multimodal Movement Prediction - Towards an Individual Assistance of Patients , 2014, PloS one.

[11]  W. Walter,et al.  Contingent Negative Variation : An Electric Sign of Sensori-Motor Association and Expectancy in the Human Brain , 1964, Nature.

[12]  Stefano Stramigioli,et al.  Stiffness and position control of a prosthetic wrist by means of an EMG interface , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[13]  Jan Christian Albiez,et al.  Towards Assistive Robotics for Home Rehabilitation , 2013, BIODEVICES.

[14]  Pieter Abbeel,et al.  Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations , 2010, 2010 IEEE International Conference on Robotics and Automation.

[15]  Mario Michael Krell,et al.  How to evaluate an agent's behavior to infrequent events?—Reliable performance estimation insensitive to class distribution , 2014, Front. Comput. Neurosci..

[16]  Nicola Vitiello,et al.  Human–Robot Synchrony: Flexible Assistance Using Adaptive Oscillators , 2011, IEEE Transactions on Biomedical Engineering.

[17]  Essa Yacoub,et al.  The Evaluation of Preprocessing Choices in Single-Subject BOLD fMRI Using NPAIRS Performance Metrics , 2003, NeuroImage.

[18]  J.J. Vidal,et al.  Real-time detection of brain events in EEG , 1977, Proceedings of the IEEE.

[19]  Fan Zhang,et al.  Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular–Mechanical Fusion , 2011, IEEE Transactions on Biomedical Engineering.

[20]  Tatsuo Narikiyo,et al.  A framework for sensorless torque estimation and control in wearable exoskeletons , 2012, 2012 12th IEEE International Workshop on Advanced Motion Control (AMC).

[21]  Jan Christian Albiez,et al.  Human Force Discrimination during Active Arm Motion for Force Feedback Design , 2013, IEEE Transactions on Haptics.

[22]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[23]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[24]  N. Thakor,et al.  Journal of Neuroengineering and Rehabilitation Open Access a Brain-computer Interface with Vibrotactile Biofeedback for Haptic Information , 2007 .

[25]  Elsa Andrea Kirchner,et al.  EMG Onset Detection - Comparison of Different Methods for a Movement Prediction Task based on EMG , 2013, BIOSIGNALS.

[26]  Rabab K Ward,et al.  A survey of signal processing algorithms in brain–computer interfaces based on electrical brain signals , 2007, Journal of neural engineering.

[27]  Mohamad Sawan,et al.  Setting Adaptive Spike Detection Threshold for Smoothed TEO Based on Robust Statistics Theory , 2012, IEEE Transactions on Biomedical Engineering.

[28]  Kun Li,et al.  Advances and Challenges in Signal Analysis for Single Trial P300-BCI , 2011, HCI.

[29]  Henry A. Nasrallah,et al.  Evoked potentials in subjects at risk for Alzheimer's Disease , 1995, Psychiatry Research.

[30]  Joachim M. Buhmann,et al.  The Balanced Accuracy and Its Posterior Distribution , 2010, 2010 20th International Conference on Pattern Recognition.

[31]  Desney S. Tan,et al.  Brain-computer interfaces for hci and games , 2008, CHI Extended Abstracts.

[32]  M. Viberg,et al.  Two decades of array signal processing research: the parametric approach , 1996, IEEE Signal Process. Mag..

[33]  Li-Wei Ko,et al.  Review of Wireless and Wearable Electroencephalogram Systems and Brain-Computer Interfaces – A Mini-Review , 2009, Gerontology.

[34]  Sirko Straube,et al.  An adaptive and efficient spatial filter for event-related potentials , 2013, 21st European Signal Processing Conference (EUSIPCO 2013).

[35]  Jon A. Mukand,et al.  Neuronal ensemble control of prosthetic devices by a human with tetraplegia , 2006, Nature.

[36]  Daniel C. Javitt,et al.  Premature disinhibition of P3 generation in schizophrenia , 1996, Biological Psychiatry.

[37]  J J Vidal,et al.  Toward direct brain-computer communication. , 1973, Annual review of biophysics and bioengineering.

[38]  Stefan Haufe,et al.  EEG potentials predict upcoming emergency brakings during simulated driving , 2011, Journal of neural engineering.

[39]  E. Hoffman,et al.  Application of annihilation coincidence detection to transaxial reconstruction tomography. , 1975, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[40]  Robert Riener,et al.  ARMin II - 7 DoF rehabilitation robot: mechanics and kinematics , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[41]  Masa-aki Sato,et al.  Visual Image Reconstruction from Human Brain Activity using a Combination of Multiscale Local Image Decoders , 2008, Neuron.

[42]  Moritz Tenorth,et al.  KnowRob: A knowledge processing infrastructure for cognition-enabled robots , 2013, Int. J. Robotics Res..

[43]  J. Wolpaw,et al.  Brain-computer communication: unlocking the locked in. , 2001, Psychological bulletin.

[44]  Jan Christian Albiez,et al.  An EMG-based Assistive Orthosis for Upper Limb Rehabilitation , 2013, BIODEVICES.

[45]  H. Lüders,et al.  American Electroencephalographic Society Guidelines for Standard Electrode Position Nomenclature , 1991, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[46]  Elsa Andrea Kirchner,et al.  A periodic spatio-spectral filter for event-related potentials , 2016, Comput. Biol. Medicine.

[47]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[48]  Frank Kirchner,et al.  Learning to Play the BRIO Labyrinth Game , 2009, Künstliche Intell..

[49]  Koby Crammer,et al.  Online Passive-Aggressive Algorithms , 2003, J. Mach. Learn. Res..

[50]  Tzyy-Ping Jung,et al.  Independent Component Analysis of Electroencephalographic Data , 1995, NIPS.

[51]  H. Berger.  Über das Elektrenkephalogramm des Menschen , 1929, Archiv für Psychiatrie und Nervenkrankheiten.

[52]  Christin Schäfer,et al.  Single Trial Detection of EEG Error Potentials: A Tool for Increasing BCI Transmission Rates , 2002, ICANN.

[53]  E. John,et al.  Evoked-Potential Correlates of Stimulus Uncertainty , 1965, Science.

[54]  Gerd Herzog,et al.  VIsual TRAnslator: Linking perceptions and natural language descriptions , 1994, Artificial Intelligence Review.

[55]  R. West.  The temporal dynamics of prospective memory: A review of the ERP and prospective memory literature , 2011, Neuropsychologia.

[56]  Dawn M. Taylor,et al.  Extracting Attempted Hand Movements from EEGs in People with Complete Hand Paralysis Following Stroke , 2011, Front. Neurosci..

[57]  Frank Kirchner,et al.  The BRIO R © Labyrinth Game-A Testbed for Reinforcement Learning and for Studies on Sensorimotor Learning , 2009 .

[58]  Uwe Meyer-Baese,et al.  Digital Signal Processing with Field Programmable Gate Arrays , 2001 .

[59]  Frank Kirchner,et al.  Assisting Telemanipulation Operators via Real-Time Brain Reading , 2009 .

[60]  E. Kirchner,et al.  PERIODIC SPATIAL FILTER FOR SINGLE TRIAL CLASSIFICATION OF EVENT RELATED BRAIN ACTIVITY , 2013, BioMed 2013.

[61]  G. McCarthy,et al.  Augmenting mental chronometry: the P300 as a measure of stimulus evaluation time. , 1977, Science.

[62]  Elsa Andrea Kirchner,et al.  Effects of eye artifact removal methods on single trial P300 detection, a comparative study , 2014, Journal of Neuroscience Methods.

[63]  Sarah J. Housman,et al.  A Randomized Controlled Trial of Gravity-Supported, Computer-Enhanced Arm Exercise for Individuals With Severe Hemiparesis , 2009, Neurorehabilitation and neural repair.

[64]  Erkki Oja,et al.  Independent Component Analysis , 2001 .

[65]  Daniel Thalmann,et al.  Improving user comfort in haptic virtual environments through gravity compensation , 2005, First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference.

[66]  P Berg,et al.  Dipole modelling of eye activity and its application to the removal of eye artefacts from the EEG and MEG. , 1991, Clinical physics and physiological measurement : an official journal of the Hospital Physicists' Association, Deutsche Gesellschaft fur Medizinische Physik and the European Federation of Organisations for Medical Physics.

[67]  Matthew A Wilson,et al.  A combinatorial method for analyzing sequential firing patterns involving an arbitrary number of neurons based on relative time order. , 2004, Journal of neurophysiology.

[68]  I Iturrate,et al.  Task-dependent signal variations in EEG error-related potentials for brain–computer interfaces , 2013, Journal of neural engineering.

[69]  B. Graimann,et al.  Why Use A BCI If You Are Healthy ? , 2000 .

[70]  P. Bifulco,et al.  A mobile EEG system with dry electrodes , 2008, 2008 IEEE Biomedical Circuits and Systems Conference.

[71]  Joseph Weizenbaum,et al.  and Machine , 1977 .

[72]  Jan Christian Albiez,et al.  A bio-inspired haptic interface for tele-robotics applications , 2009, 2008 IEEE International Conference on Robotics and Biomimetics.

[73]  Ben H. Jansen,et al.  An exploratory study of factors affecting single trial P300 detection , 2004, IEEE Transactions on Biomedical Engineering.

[74]  Frank Kirchner,et al.  Conceptual and Embodiment Design of Robotic Prototypes , 2011, Int. J. Humanoid Robotics.

[75]  Jürgen Schmidhuber,et al.  A System for Robotic Heart Surgery that Learns to Tie Knots Using Recurrent Neural Networks , 2006 .

[76]  Thai Son Hoang,et al.  Rodin: an open toolset for modelling and reasoning in Event-B , 2010, International Journal on Software Tools for Technology Transfer.

[77]  G. Edlinger,et al.  Laboratory PC and Mobile Pocket PC Brain-Computer Interface Architectures , 2005, 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference.

[78]  A. Kheddar,et al.  fMRI robotic embodiment: A pilot study , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[79]  A. Graser,et al.  Brain-Computer Interface for high-level control of rehabilitation robotic systems , 2007, 2007 IEEE 10th International Conference on Rehabilitation Robotics.

[80]  A. Hodgkin,et al.  Action Potentials Recorded from Inside a Nerve Fibre , 1939, Nature.

[81]  H. Kornhuber,et al.  Distribution of readiness potential, pre-motion positivity, and motor potential of the human cerebral cortex preceding voluntary finger movements , 2004, Experimental Brain Research.

[82]  N. Birbaumer.  Breaking the silence: brain-computer interfaces (BCI) for communication and motor control. , 2006, Psychophysiology.

[83]  Benjamin Blankertz,et al.  THE BERLIN BRAIN-COMPUTER INTERFACE PRESENTS THE NOVEL MENTAL TYPEWRITER HEX-O-SPELL , 2006 .

[84]  N. Squires,et al.  Two varieties of long-latency positive waves evoked by unpredictable auditory stimuli in man. , 1975, Electroencephalography and clinical neurophysiology.

[85]  Boris Reuderink,et al.  Games and Brain-Computer Interfaces: The State of the Art , 2008 .

[86]  T. Gasser,et al.  Correction of EOG artifacts in event-related potentials of the EEG: aspects of reliability and validity. , 1982, Psychophysiology.

[87]  Robert Wille,et al.  Completeness-Driven Development , 2012, ICGT.

[88]  Stefan Kopp,et al.  Embodied Gesture Processing: Motor-Based Integration of Perception and Action in Social Artificial Agents , 2010, Cognitive Computation.

[89]  Frank Kirchner,et al.  A Connectionist Architecture for Learning to Play a Simulated Brio Labyrinth Game , 2007, KI.

[90]  F. Freeman,et al.  Evaluation of an adaptive automation system using three EEG indices with a visual tracking task , 1999, Biological Psychology.

[91]  Adrian D. C. Chan,et al.  Flexible dry electrode for recording surface electromyogram , 2010, 2010 IEEE Instrumentation & Measurement Technology Conference Proceedings.

[92]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[93]  J. Millán,et al.  Detection of self-paced reaching movement intention from EEG signals , 2012, Front. Neuroeng..

[94]  Elmar Nöth,et al.  VERBMOBIL: the use of prosody in the linguistic components of a speech understanding system , 2000, IEEE Trans. Speech Audio Process..

[95]  Maja J. Matarić,et al.  Learning Motor Skills by Imitation , 1994, AAAI 1994.

[96]  Desney S. Tan,et al.  Feasibility and pragmatics of classifying working memory load with an electroencephalograph , 2008, CHI.

[97]  O. Arthurs,et al.  How well do we understand the neural origins of the fMRI BOLD signal? , 2002, Trends in Neurosciences.

[98]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[99]  Wensheng Xu,et al.  A Robot Control System Based on Gesture Recognition Using Kinect , 2013 .

[100]  Bruce A. MacDonald,et al.  Facial Expression Recognition for Human-Robot Interaction - A Prototype , 2008, RobVis.

[101]  Klaus-Robert Müller,et al.  Towards Zero Training for Brain-Computer Interfacing , 2008, PloS one.

[102]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[103]  Yasuhisa Hasegawa,et al.  Intention-based walking support for paraplegia patients with Robot Suit HAL , 2007, Adv. Robotics.

[104]  Feng Wan,et al.  Classification of Single-Trial EEG Based on Support Vector Clustering during Finger Movement , 2009, ISNN.

[105]  M. Coles.  Modern mind-brain reading: psychophysiology, physiology, and cognition. , 1989, Psychophysiology.

[106]  N. Birbaumer,et al.  Brain-computer communication: self-regulation of slow cortical potentials for verbal communication. , 2001, Archives of physical medicine and rehabilitation.

[107]  Eric J Perreault,et al.  Decoding with limited neural data: a mixture of time-warped trajectory models for directional reaches , 2012, Journal of neural engineering.

[108]  Moshe Y. Vardi,et al.  A Temporal Language for SystemC , 2008, 2008 Formal Methods in Computer-Aided Design.

[109]  H. Kazerooni,et al.  Biomechanical design of the Berkeley lower extremity exoskeleton (BLEEX) , 2006, IEEE/ASME Transactions on Mechatronics.

[110]  Hendrik Wöhrle,et al.  Online Detection of P300 related Target Recognition Processes During a Demanding Teleoperation Task - Classifier Transfer for the Detection of Missed Targets , 2014, PhyCS.

[111]  Thorsten O. Zander,et al.  Enhancing Human-Computer Interaction with Input from Active and Passive Brain-Computer Interfaces , 2010, Brain-Computer Interfaces.

[112]  P. Sajda,et al.  Response error correction-a demonstration of improved human-machine performance using real-time EEG monitoring , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[113]  Elsa Andrea Kirchner,et al.  Minimizing Calibration Time for Brain Reading , 2011, DAGM-Symposium.

[114]  M. Kliegel,et al.  The role of dual-task and task-switch in prospective memory: Behavioural data and neural correlates , 2009, Neuropsychologia.

[115]  C. C. Duncan,et al.  Event-related brain potentials: a window on information processing in schizophrenia. , 1988, Schizophrenia bulletin.

[116]  J.D. Bayliss,et al.  Use of the evoked potential P3 component for control in a virtual apartment , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[117]  Edward Cutrell,et al.  BCI for passive input in HCI , 2007 .

[118]  A. Kübler,et al.  ERPs contributing to classification in the ”P300” BCI , 2011 .

[119]  G. Pfurtscheller,et al.  Brain motor system function in a patient with complete spinal cord injury following extensive brain–computer interface training , 2008, Experimental Brain Research.

[120]  Stephan Merz,et al.  Model Checking , 2000 .

[121]  Jacob Rosen,et al.  A myosignal-based powered exoskeleton system , 2001, IEEE Trans. Syst. Man Cybern. Part A.

[122]  Klaus-Robert Müller,et al.  Machine-Learning Based Co-adaptive Calibration: A Perspective to Fight BCI Illiteracy , 2010, HAIS.

[123]  Frank Kirchner,et al.  A Framework for High Performance Embedded Signal Processing and Classification of Psychophysiological Data , 2013 .

[124]  Luc Van Gool,et al.  Real-time 3D hand gesture interaction with a robot for understanding directions from humans , 2011, 2011 RO-MAN.

[125]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[126]  Michela Balconi,et al.  The multicomponential nature of movement-related cortical potentials: functional generators and psychological factors , 2009 .

[127]  A. Kok.  Event-related-potential (ERP) reflections of mental resource̊s: a review and synthesis , 1997, Biological Psychology.

[128]  Hiroshi Sawada,et al.  Direction of arrival estimation for multiple source signals using independent component analysis , 2003, Seventh International Symposium on Signal Processing and Its Applications, 2003. Proceedings..

[129]  Jan Christian Albiez,et al.  A Multimodal Brain-arm Interface for Operation of Complex Robotic Systems and Upper Limb Motor Recovery , 2011, BIODEVICES.

[130]  Peter Stone,et al.  Transfer Learning for Reinforcement Learning Domains: A Survey , 2009, J. Mach. Learn. Res..

[131]  Marina Schmid,et al.  An Introduction To The Event Related Potential Technique , 2016 .

[132]  Nicholas Wymbs,et al.  Effects of intention load and background context on prospective remembering: an event-related brain potential study. , 2003, Psychophysiology.

[133]  David D. Woods,et al.  Decomposing Automation : Apparent Simplicity , Real Complexity , 2003 .

[134]  R. McCarley,et al.  Button-pressing affects P300 amplitude and scalp topography , 2001, Clinical Neurophysiology.

[135]  Elsa Andrea Kirchner,et al.  Exoskeleton Technology in Rehabilitation: Towards an EMG-Based Orthosis System for Upper Limb Neuromotor Rehabilitation , 2013, J. Robotics.

[136]  Eduardo Rocon de Lima,et al.  An EEG-Based Design for the Online Detection of Movement Intention , 2011, IWANN.

[137]  Wolfgang Wahlster,et al.  Mobile Speech-to-Speech Translation of Spontaneous Dialogs: An Overview of the Final Verbmobil System , 2000 .

[138]  J. Weizenbaum.  From Computer Power and Human Reason From Judgment to Calculation , 2007 .

[139]  R. Johnson.  A triarchic model of P300 amplitude. , 1986, Psychophysiology.

[140]  Brendan Z. Allison,et al.  The Hybrid BCI , 2010, Frontiers in Neuroscience.

[141]  Stephen H. Fairclough,et al.  A research agenda for physiological computing , 2004, Interact. Comput..

[142]  M. Wilson,et al.  Temporally Structured Replay of Awake Hippocampal Ensemble Activity during Rapid Eye Movement Sleep , 2001, Neuron.

[143]  Evgueniy V. Lubenov,et al.  Prefrontal Phase Locking to Hippocampal Theta Oscillations , 2005, Neuron.

[144]  Anatole Lécuyer,et al.  An overview of research on "passive" brain-computer interfaces for implicit human-computer interaction , 2010 .

[145]  M. Fahle,et al.  On the Applicability of Brain Reading for Predictive Human-Machine Interfaces in Robotics , 2013, PloS one.

[146]  David Feess,et al.  Comparison of Sensor Selection Mechanisms for an ERP-Based Brain-Computer Interface , 2013, PloS one.

[147]  Miloslav Kukleta,et al.  Intracerebral recordings of the Bereitschaftspotential demonstrate the heterogeneity of its components. , 2012, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[148]  K. Cheung.  Implantable microscale neural interfaces , 2007, Biomedical microdevices.

[149]  Christopher M. Bishop,et al.  Pattern Recognition and Machine Learning (Information Science and Statistics) , 2006 .

[150]  Joel C. Perry,et al.  Real-Time Myoprocessors for a Neural Controlled Powered Exoskeleton Arm , 2006, IEEE Transactions on Biomedical Engineering.

[151]  Susan Karlin.  Raiding Iron Man's closet [Geek Life] , 2011, IEEE Spectrum.

[152]  A. Kok.  On the utility of P3 amplitude as a measure of processing capacity. , 2001, Psychophysiology.

[153]  Ethan R. Buch,et al.  Physiological regulation of thinking: brain-computer interface (BCI) research. , 2006, Progress in brain research.

[154]  B. Libet,et al.  Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential). The unconscious initiation of a freely voluntary act. , 1983, Brain : a journal of neurology.

[155]  Bilge Mutlu,et al.  Modeling and Evaluating Narrative Gestures for Humanlike Robots , 2013, Robotics: Science and Systems.

[156]  E Donchin,et al.  The truth will out: interrogative polygraphy ("lie detection") with event-related brain potentials. , 1991, Psychophysiology.

[157]  Russell Tessier,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. Reconfigurable Computing for Digital Signal Processing: A Survey ∗ , 1999 .

[158]  Klaus-Robert Müller,et al.  The Berlin Brain-Computer Interface: Machine Learning Based Detection of User Specific Brain States , 2006, J. Univers. Comput. Sci..

[159]  R. Verleger,et al.  Reduction of P3b in patients with temporo-parietal lesions. , 1994, Brain research. Cognitive brain research.

[160]  Patrick Suppes,et al.  Partial Orders of Similarity Differences Invariant Between EEG-Recorded Brain and Perceptual Representations of Language , 2009, Neural Computation.

[161]  Paul-Gerhard Plöger,et al.  Facial Expression Recognition for Domestic Service Robots , 2011, RoboCup.

[162]  Heinz Wörn,et al.  An intelligent and autonomous endoscopic guidance system for minimally invasive surgery , 2011, 2011 IEEE International Conference on Robotics and Automation.

[163]  Stephen J. Roberts,et al.  Adaptive Classification by Hybrid EKF with Truncated Filtering: Brain Computer Interfacing , 2008, IDEAL.

[164]  Lüder Deecke,et al.  Voluntary finger movement in man: Cerebral potentials and theory , 1976, Biological Cybernetics.

[165]  Rolf Drechsler,et al.  System level validation using formal techniques , 2005 .

[166]  Alexei Makarenko,et al.  Human-robot communication for collaborative decision making - A probabilistic approach , 2010, Robotics Auton. Syst..

[167]  Niels Birbaumer,et al.  fMRI Brain-Computer Interface: A Tool for Neuroscientific Research and Treatment , 2007, Comput. Intell. Neurosci..

[168]  N. Birbaumer,et al.  BCI2000: a general-purpose brain-computer interface (BCI) system , 2004, IEEE Transactions on Biomedical Engineering.

[169]  J. Polich.  Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[170]  R. Verleger.  On the utility of P3 latency as an index of mental chronometry. , 1997, Psychophysiology.

[171]  H. Moldofsky,et al.  A spectral method for removing eye movement artifacts from the EEG. , 1978, Electroencephalography and clinical neurophysiology.

[172]  F. Kirchner,et al.  Using Kinematically Complex Robots for Case Studies in Embodied Cognition , 2006 .

[173]  Wolfram Burgard,et al.  Probabilistic Robotics (Intelligent Robotics and Autonomous Agents) , 2005 .

[174]  Christa Neuper,et al.  Walking by Thinking: The Brainwaves Are Crucial, Not the Muscles! , 2006, PRESENCE: Teleoperators and Virtual Environments.

[175]  Andreas Wege,et al.  Application of EMG signals for controlling exoskeleton robots , 2006, Biomedizinische Technik. Biomedical engineering.

[176]  Kai Ming Ting,et al.  Confusion Matrix , 2010, Encyclopedia of Machine Learning and Data Mining.

[177]  Po-Lei Lee,et al.  Development of a Low-Cost FPGA-Based SSVEP BCI Multimedia Control System , 2010, IEEE Transactions on Biomedical Circuits and Systems.

[178]  T W Picton,et al.  The P300 Wave of the Human Event‐Related Potential , 1992, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[179]  D. Feess,et al.  Looking at ERPs from Another Perspective: Polynomial Feature Analysis , 2013 .

[180]  N. Logothetis.  The neural basis of the blood-oxygen-level-dependent functional magnetic resonance imaging signal. , 2002, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[181]  Salil H. Patel,et al.  Characterization of N200 and P300: Selected Studies of the Event-Related Potential , 2005, International journal of medical sciences.

[182]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[183]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[184]  Mark S. Schwartz,et al.  Biofeedback: A Practitioner's Guide , 1987 .

[185]  E. Donchin,et al.  P300 and tracking difficulty: evidence for multiple resources in dual-task performance. , 1980, Psychophysiology.

[186]  Manfred Fahle,et al.  EEG in Dual-Task Human-Machine Interaction: On the Feasibility of EEG based Support of Complex Human-Machine Interaction , 2013 .

[187]  Cuntai Guan,et al.  Learning from other subjects helps reducing Brain-Computer Interface calibration time , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[188]  G. R. Muller,et al.  Brain oscillations control hand orthosis in a tetraplegic , 2000, Neuroscience Letters.

[189]  Robert Riener,et al.  Predicting Targets of Human Reaching Motions Using Different Sensing Technologies , 2013, IEEE Transactions on Biomedical Engineering.

[190]  A. Palmini,et al.  The concept of the epileptogenic zone: a modern look at Penfield and Jasper's views on the role of interictal spikes. , 2006, Epileptic disorders : international epilepsy journal with videotape.

[191]  B. McNaughton,et al.  Tetrodes markedly improve the reliability and yield of multiple single-unit isolation from multi-unit recordings in cat striate cortex , 1995, Journal of Neuroscience Methods.

[192]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[193]  Elsa Andrea Kirchner,et al.  Classifier Transferability in the Detection of Error Related Potentials from Observation to Interaction , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[194]  M. Simonetta,et al.  Bereitschaftspotential in a simple movement or in a motor sequence starting with the same simple movement. , 1991, Electroencephalography and clinical neurophysiology.

[195]  Tim Bowman.  VR meets physical therapy , 1997, CACM.

[196]  Andy Ruina,et al.  A Bipedal Walking Robot with Efficient and Human-Like Gait , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[197]  Teodiano Freire Bastos Filho,et al.  An embedded system for an EEG based BCI , 2013, 2013 ISSNIP Biosignals and Biorobotics Conference: Biosignals and Robotics for Better and Safer Living (BRC).

[198]  Z J Koles,et al.  The quantitative extraction and topographic mapping of the abnormal components in the clinical EEG. , 1991, Electroencephalography and clinical neurophysiology.

[199]  Mario Michael Krell,et al.  pySPACE—a signal processing and classification environment in Python , 2013, Front. Neuroinform..

[200]  D. Cohen.  Magnetoencephalography: Evidence of Magnetic Fields Produced by Alpha-Rhythm Currents , 1968, Science.

[201]  István Ulbert,et al.  Multiple microelectrode-recording system for human intracortical applications , 2001, Journal of Neuroscience Methods.

[202]  Frank Kirchner,et al.  Towards Operator Monitoring via Brain Reading - An EEG-based Approach for Space Applications , 2010 .

[203]  Keijiro Yamamoto,et al.  Development of Power Assisting Suit : Miniaturization of Supply System to Realize Wearable Suit( Advances in Motion and Vibration Control Technology) , 2003 .

[204]  Nicholas Wymbs,et al.  Is detecting prospective cues the same as selecting targets? An ERP study , 2004, Cognitive, affective & behavioral neuroscience.

[205]  K. Yamazaki,et al.  Enhanced negative slope of the readiness potential preceding a target force production task. , 1998, Electroencephalography and clinical neurophysiology.

[206]  Otto Muzik,et al.  Origin and Propagation of Epileptic Spasms Delineated on Electrocorticography , 2005, Epilepsia.

[207]  B. Blankertz,et al.  Unsupervised adaptation of the LDA classifier for Brain-Computer interfaces , 2008 .

[208]  J. Hanley,et al.  The meaning and use of the area under a receiver operating characteristic (ROC) curve. , 1982, Radiology.

[209]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[210]  M. Baudry,et al.  Advances in Network Electrophysiology , 2006 .

[211]  Alexander Fabisch,et al.  Towards Learning of Generic Skills for Robotic Manipulation , 2014, KI - Künstliche Intelligenz.

[212]  Stephen C. Strother,et al.  Support vector machines for temporal classification of block design fMRI data , 2005, NeuroImage.

[213]  Michele Folgheraiter,et al.  Measuring the Improvement of the Interaction Comfort of a Wearable Exoskeleton , 2012, Int. J. Soc. Robotics.

[214]  Niko Wilbert,et al.  Modular Toolkit for Data Processing (MDP): A Python Data Processing Framework , 2008, Frontiers Neuroinformatics.

[215]  Dario Izzo,et al.  Brain machine interfaces for space applications : enhancing astronaut capabilities , 2009 .

[216]  Klaus-Robert Müller,et al.  Subject-independent mental state classification in single trials , 2009, Neural Networks.

[217]  Haris Dindo,et al.  A probabilistic approach to learning a visually grounded language model through human-robot interaction , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[218]  Rolf Drechsler,et al.  A formal model for embedded brain reading , 2013, Ind. Robot.

[219]  Martin Eimer,et al.  The lateralized readiness potential as an on-line measure of central response activation processes , 1998 .

[220]  Stefan Kopp,et al.  Modeling the Semantic Coordination of Speech and Gesture under Cognitive and Linguistic Constraints , 2013, IVA.

[221]  H. Kornhuber,et al.  Hirnpotentialänderungen bei Willkürbewegungen und passiven Bewegungen des Menschen: Bereitschaftspotential und reafferente Potentiale , 1965, Pflüger's Archiv für die gesamte Physiologie des Menschen und der Tiere.

[222]  Frank Kirchner,et al.  Using Embodied Data for Localization and Mapping , 2014, J. Field Robotics.

[223]  S. Hillyard,et al.  Eye movement artifact in the CNV. , 1970, Electroencephalography and clinical neurophysiology.

[224]  Guillermo Paradiso,et al.  Involvement of human thalamus in the preparation of self-paced movement. , 2004, Brain : a journal of neurology.

[225]  H Shibasaki,et al.  Components of the movement-related cortical potential and their scalp topography. , 1980, Electroencephalography and clinical neurophysiology.

[226]  M. Hallett,et al.  Prediction of human voluntary movement before it occurs , 2011, Clinical Neurophysiology.

[227]  H Shibasaki,et al.  Enhanced negative slope of cortical potentials before sequential as compared with simultaneous extensions of two fingers. , 1993, Electroencephalography and clinical neurophysiology.

[228]  Sirko Straube,et al.  Striving for Better and Earlier Movement Prediction by Postprocessing of Classification Scores , 2013, NEUROTECHNIX.

[229]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[230]  R. Homan,et al.  Cerebral location of international 10-20 system electrode placement. , 1987, Electroencephalography and clinical neurophysiology.

[231]  Yoshiyuki Sankai,et al.  Control method of robot suit HAL working as operator's muscle using biological and dynamical information , 2005, 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[232]  Thorsten O. Zander,et al.  Utilizing Secondary Input from Passive Brain-Computer Interfaces for Enhancing Human-Machine Interaction , 2009, HCI.

[233]  Frank Kirchner,et al.  Exploiting Sensorimotor Coordination for Learning to Recognize Objects , 2007, IJCAI.

[234]  G R Müller-Putz,et al.  Toward smarter BCIs: extending BCIs through hybridization and intelligent control , 2012, Journal of neural engineering.

[235]  Mario Michael Krell,et al.  A Dataflow-based Mobile Brain Reading System on Chip with Supervised Online Calibration - For Usage without Acquisition of Training Data , 2016, NEUROTECHNIX.

[236]  M. Hallett,et al.  What is the Bereitschaftspotential? , 2006, Clinical Neurophysiology.

[237]  K.-R. Muller,et al.  The Berlin brain-computer interface: EEG-based communication without subject training , 2006, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[238]  David H. Barlow,et al.  Abnormal Psychology: An Integrative Approach , 1994 .

[239]  G. Rees,et al.  Neuroimaging: Decoding mental states from brain activity in humans , 2006, Nature Reviews Neuroscience.

[240]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[241]  Elsa Andrea Kirchner,et al.  Rapid Adaptation of Brain Reading Interfaces based on Threshold Adjustment , 2010 .

[242]  E. Donchin,et al.  Detecting early communication: Using measures of movement-related potentials to illuminate human information processing , 1988, Biological Psychology.

[243]  E Hennighausen,et al.  Missed prime words within the attentional blink evoke an N400 semantic priming effect. , 2001, Psychophysiology.

[244]  Ho-Sub Yoon,et al.  Vision-based arm gesture recognition for a long-range human–robot interaction , 2010, The Journal of Supercomputing.

[245]  Sirko Straube,et al.  Online movement prediction in a robotic application scenario , 2013, 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER).

[246]  Kristinn R. Thórisson,et al.  Learning Problem Solving Skills from Demonstration: An Architectural Approach , 2011, AGI.

[247]  Foad Ghaderi.  Joint spatial and spectral filter estimation for single trial detection of event related potentials , 2013, 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP).

[248]  G. Pfurtscheller,et al.  Prosthetic Control by an EEG-based Brain-Computer Interface (BCI) , 2001 .

[249]  E. Donchin,et al.  Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. , 1988, Electroencephalography and clinical neurophysiology.