Human-machine Interface in Multimedia Communication Examples of Revolutions and Evolutions in Communications an Overview of Bcis

Human-computer interface (HCI) has been a growing field of research and development in recent years [1]-[4]. Most of the effort has been dedicated to the design of user-friendly and ergonomic systems by means of innovative interfaces such as voice, vision, and other input/output devices in virtual reality [5]-[15]. Direct brain-computer interface (BCI) adds a new dimension to HCI [16]-[23]. Interesting research in this direction has already been initiated, motivated by the hope of creating new communication channels for persons with severe motor disabilities. In this article, we approach the problem of BCI from the viewpoint of interactions in a multimedia-rich environment for the general consumer market. However, this is by no means incompatible with applications for motor impaired subjects. There is a general consensus that BCI represents a new frontier in science and technology. One of the challenging aspects is the need for multidisciplinary skills to achieve this goal. The growing field of BCI is in its infancy, and a significant amount of research is still needed to answer many questions and to resolve many complex problems. This article raises various issues in the design of an efficient BCI system in multimedia applications. The main focus will be on one specific modality, namely electroencephalography (EEG)-based BCI. In doTouradj Ebrahimi, Jean-Marc Vesin, and Gary Garcia

[1]  E Donchin,et al.  The mental prosthesis: assessing the speed of a P300-based brain-computer interface. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[2]  G Pfurtscheller,et al.  Real-time EEG analysis with subject-specific spatial patterns for a brain-computer interface (BCI). , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[3]  G Calhoun,et al.  Brain-computer interfaces based on the steady-state visual-evoked response. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[4]  C.W. Anderson,et al.  Multivariate autoregressive models for classification of spontaneous electroencephalographic signals during mental tasks , 1998, IEEE Transactions on Biomedical Engineering.

[5]  Febo Cincotti,et al.  Relevant EEG features for the classification of spontaneous motor-related tasks , 2002, Biological Cybernetics.

[6]  H. Jasper,et al.  The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. , 1999, Electroencephalography and clinical neurophysiology. Supplement.

[7]  J.J. Vidal,et al.  Real-time detection of brain events in EEG , 1977, Proceedings of the IEEE.

[8]  E Donchin,et al.  Brain-computer interface technology: a review of the first international meeting. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[9]  J.A. Landay.  Informal User Interface For Natural Human-Computer Interaction , 1998, IEEE Intelligent Systems and their Applications.

[10]  T. Lagerlund,et al.  Spatial filtering of multichannel electroencephalographic recordings through principal component analysis by singular value decomposition. , 1997, Journal of clinical neurophysiology : official publication of the American Electroencephalographic Society.

[11]  Erich E. Sutter,et al.  The brain response interface: communication through visually-induced electrical brain responses , 1992 .

[12]  Vladimir Pavlovic,et al.  Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[14]  G. Pfurtscheller,et al.  Information transfer rate in a five-classes brain-computer interface , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[15]  H. Flor,et al.  The thought translation device (TTD) for completely paralyzed patients. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[16]  G Pfurtscheller,et al.  Current trends in Graz Brain-Computer Interface (BCI) research. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[17]  Eric Badiqué.  New imaging frontiers: 3D and mixed reality , 2002, Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission.

[18]  G E Birch,et al.  Brain-computer interface research at the Neil Squire Foundation. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[19]  J. Wolpaw,et al.  Brain-computer communication: unlocking the locked in. , 2001, Psychological bulletin.

[20]  Rainer Stiefelhagen,et al.  Gaze tracking for multimodal human-computer interaction , 1997, 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[21]  Tzyy-Ping Jung,et al.  Imaging brain dynamics using independent component analysis , 2001, Proc. IEEE.

[22]  G Pfurtscheller,et al.  Frequency component selection for an EEG-based brain to computer interface. , 1999, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[23]  D. Tucker,et al.  EEG coherency. I: Statistics, reference electrode, volume conduction, Laplacians, cortical imaging, and interpretation at multiple scales. , 1997, Electroencephalography and clinical neurophysiology.

[24]  Jakub Segen,et al.  Human-computer interaction using gesture recognition and 3D hand tracking , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[25]  J. Wolpaw,et al.  Multichannel EEG-based brain-computer communication. , 1994, Electroencephalography and clinical neurophysiology.

[26]  Arnold Neumaier,et al.  Estimation of parameters and eigenmodes of multivariate autoregressive models , 2001, TOMS.

[27]  Dominic W. Massaro.  Perceptual interfaces in human computer interaction , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[28]  L P Panych,et al.  Practical digital filters for reducing EMG artefact in EEG seizure recordings. , 1989, Electroencephalography and clinical neurophysiology.

[29]  C. W. Therrien,et al.  Decision, Estimation and Classification: An Introduction to Pattern Recognition and Related Topics , 1989 .

[30]  Ying Wu,et al.  Nonstationary color tracking for vision-based human-computer interaction , 2002, IEEE Trans. Neural Networks.

[31]  Gary E. Birch,et al.  A brain-controlled switch for asynchronous control applications , 2000, IEEE Trans. Biomed. Eng..

[32]  B. Oken.  Electrophysiology of Mind: Event-Related Brain Potentials and Cognition , 1996 .

[33]  Hideyuki Tamura,et al.  Mixed Reality: Future Dreams Seen at the Border between Real and Virtual Worlds , 2001, IEEE Computer Graphics and Applications.

[34]  Jen Grey,et al.  "Human-computer interaction in life drawing, a fine artist's perspective" , 2002, Proceedings Sixth International Conference on Information Visualisation.

[35]  G. Pfurtscheller,et al.  Optimal spatial filtering of single trial EEG during imagined hand movement. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[36]  Kazuhiko Yamamoto,et al.  Face and hand gesture recognition for human-computer interaction , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[37]  C. Martyn Jones,et al.  Human-computer interaction and animation system for simple interfacing to virtual environments , 1997 .

[38]  G. Pfurtscheller,et al.  Rapid prototyping of an EEG-based brain-computer interface (BCI) , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[39]  Gert Pfurtscheller,et al.  Motor imagery and direct brain-computer communication , 2001, Proc. IEEE.

[40]  J D Bayliss,et al.  A virtual reality testbed for brain-computer interface research. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[41]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[42]  Manuel Davy.  Noyaux optimisés pour la classification dans le plan temps-fréquence : proposition d'un alorithme constructif et d'une référence bayésienne basée sur les méthodes MCMC : application au diagnostic d'enceintes acoustiques , 2000 .

[43]  D J McFarland,et al.  Brain-computer interface research at the Wadsworth Center. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[44]  G. Pfurtscheller,et al.  Brain-Computer Interfaces for Communication and Control. , 2011, Communications of the ACM.

[45]  T. Sejnowski,et al.  Removing electroencephalographic artifacts by blind source separation. , 2000, Psychophysiology.

[46]  Maximilian Reiser,et al.  Virtual reality and multimedia human-computer interaction in medicine , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[47]  Z. Keirn,et al.  A new mode of communication between man and his surroundings , 1990, IEEE Transactions on Biomedical Engineering.

[48]  Touradj Ebrahimi,et al.  Time-Frequency-Space Kernel for Single EEG-Trial Classification , 2002 .

[49]  Jukka Heikkonen,et al.  A local neural classifier for the recognition of EEG patterns associated to mental tasks , 2002, IEEE Trans. Neural Networks.

[50]  Yousef H. Daabaj.  An evaluation of the usability of human-computer interaction methods in support of the development of interactive systems , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[51]  Erkki Oja,et al.  Independent component analysis: algorithms and applications , 2000, Neural Networks.

[52]  Andrzej Cichocki,et al.  Blind noise reduction for multisensory signals using ICA and subspace filtering, with application to EEG analysis , 2002, Biological Cybernetics.

[53]  G Pfurtscheller,et al.  Using time-dependent neural networks for EEG classification. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[54]  Thomas S. Huang,et al.  Emotional expressions in audiovisual human computer interaction , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[55]  Erkki Oja,et al.  Independent component approach to the analysis of EEG and MEG recordings , 2000, IEEE Transactions on Biomedical Engineering.

[56]  Te-Won Lee,et al.  Independent Component Analysis , 1998, Springer US.

[57]  Shogo Nishida,et al.  An analysis of EEG based on information flow with SD method , 1998, SMC'98 Conference Proceedings. 1998 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.98CH36218).

[58]  M J Stokes,et al.  EEG-based communication: a pattern recognition approach. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[59]  H. Yamamoto.  Case studies of producing mixed reality worlds , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[60]  Rosalind W. Picard.  Affective Computing , 1997 .

[61]  Joseph D. Bronzino,et al.  Principles of Electroencephalography , 2014 .

[62]  J.-M. Vesin,et al.  Classification of EEG signals in the ambiguity domain for brain computer interface applications , 2002, 2002 14th International Conference on Digital Signal Processing Proceedings. DSP 2002 (Cat. No.02TH8628).

[63]  Jasha Droppo,et al.  Optimizing time-frequency distributions for automatic classification , 1997, Optics & Photonics.

[64]  Christian D. Schunn,et al.  Integrating perceptual and cognitive modeling for adaptive and intelligent human-computer interaction , 2002, Proc. IEEE.

[65]  L. Vigon,et al.  Quantitative evaluation of techniques for ocular artefact filtering of EEG waveforms , 2000 .

[66]  Mark Coates.  Time frequency modelling , 1999 .

[67]  N. Kaiyan.  Exploratory study of implicit theories in human computer interaction , 1996, Proceedings Sixth Australian Conference on Computer-Human Interaction.