Synchronous stereo-video and biosignal recording - a basic setup for Human-Computer-Interface applications

The work presented reports on the development of a data acquisition system intended to be used in human-computer-interfacing applications to synchronously record upper limb movement and corresponding biosignals, be it EEG, ECoG, EMG or multi unit neuronal signals. Ultimately, this aims at the clinical recording of ECoG signals from awake patients undergoing tumor resection. For that purpose stereo-video camera frames are used to detect and triangulate hand and finger positions with high precision while at the same time a DSP-based 32-channel board acquires wideband biosignals. We validated the synchronous acquisition of both space coordinates and bioelectrical signals (EMG) by performing simple grasp experiments. Further research will be dedicated to the system's clinical application

[1]  D.M. Taylor,et al.  Information conveyed through brain-control: cursor versus robot , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[2]  Dieter Schmalstieg,et al.  Finger tracking for interaction in augmented environments , 2001, Proceedings IEEE and ACM International Symposium on Augmented Reality.

[3]  C. Jennings,et al.  Robust finger tracking with multiple cameras , 1999, Proceedings International Workshop on Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems. In Conjunction with ICCV'99 (Cat. No.PR00378).

[4]  C. Mehring,et al.  Inference of hand movements from local field potentials in monkey motor cortex , 2003, Nature Neuroscience.

[5]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[6]  Winnie Jensen,et al.  A Novel High Channel-Count System for Acute Multisite Neuronal Recordings , 2006, IEEE Transactions on Biomedical Engineering.

[7]  O. Suess,et al.  Impact of Brain Shift on Intraoperative Neurophysiological Monitoring with Cortical Strip Electrodes , 2002, Acta Neurochirurgica.

[8]  Jakub Segen,et al.  Shadow gestures: 3D hand pose estimation using a single camera , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[9]  Hideo Saito,et al.  3-D drawing system via hand motion recognition from two cameras , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[10]  U G Hofmann,et al.  A 64(128)-CHANNEL MULTISITE NEURONAL RECORDING SYSTEM , 2002, Biomedizinische Technik. Biomedical engineering.

[11]  Miguel A. L. Nicolelis,et al.  Actions from thoughts , 2001, Nature.

[12]  S. Meagher Instant neural control of a movement signal , 2002 .

[13]  M. Grabois,et al.  Locked-in syndrome: a review of 139 cases. , 1986, Stroke.

[14]  Tonio Ball,et al.  Towards an implantable brain-machine interface based on epicortical field potentials , 2004 .

[15]  Takeo Kanade,et al.  DigitEyes: vision-based hand tracking for human-computer interaction , 1994, Proceedings of 1994 IEEE Workshop on Motion of Non-rigid and Articulated Objects.

[16]  P. Black The present and future of cerebral tumor surgery in children , 2000, Child's Nervous System.

[17]  D. Humphrey,et al.  Pattern Recognition Techniques for Cortical Control of a Robotic Arm , 2007 .

[18]  Björn Stenger,et al.  Hand Pose Estimation Using Hierarchical Detection , 2004, ECCV Workshop on HCI.

[19]  Takeo Kanade,et al.  DigitEyes: Vision-Based Human Hand Tracking , 1993 .

[20]  Gerwin Schalk,et al.  A brain–computer interface using electrocorticographic signals in humans , 2004, Journal of neural engineering.

[21]  M. Laubach,et al.  Cortical ensemble activity increasingly predicts behaviour outcomes during learning of a motor task , 2022 .

[22]  D J McFarland,et al.  An EEG-based brain-computer interface for cursor control. , 1991, Electroencephalography and clinical neurophysiology.

[23]  Miguel A. L. Nicolelis,et al.  Real-time control of a robot arm using simultaneously recorded neurons in the motor cortex , 1999, Nature Neuroscience.

[24]  Melody Moore,et al.  Using human extra-cortical local field potentials to control a switch. , 2004, Journal of neural engineering.

[25]  S. Ahmad,et al.  A usable real-time 3D hand tracker , 1994, Proceedings of 1994 28th Asilomar Conference on Signals, Systems and Computers.

[26]  Emanuele Trucco,et al.  Introductory techniques for 3-D computer vision , 1998 .

[27]  H. Flor,et al.  The thought translation device (TTD) for completely paralyzed patients. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[28]  Yoichi Sato,et al.  Fast tracking of hands and fingertips in infrared images for augmented desk interface , 2000, Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580).

[29]  P. Kennedy,et al.  The cone electrode: Ultrastructural studies following long-term recording in rat and monkey cortex , 1992, Neuroscience Letters.

[30]  Gunilla Borgefors,et al.  Distance transformations in digital images , 1986, Comput. Vis. Graph. Image Process..

[31]  James L. Crowley,et al.  Active hand tracking , 1998, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition.