Human-Computer Interface Technologies for the Motor Impaired

Introduction Abstract Introduction: Human-computer interface for people with disabilities Background History Future of HCI Layout of the book Reference Human-computer interface: Mechanical sensors Abstract Introduction Modified devices Sensors Applications of HCI based on mechanical sensors Current research and future improvements References Brain-computer interface based on thought waves Abstract Introduction History of brain-computer interface Significance of BCI devices BCI technology System design Signal analysis BCI translation algorithms User consideration Applications of BCI Limitations Future research Ethical consideration References Evoked potentials-based brain-computer interface Abstract Introduction Brain-computer interface (BCI) systems based on steady-state visual evoke potential Design challenges and limitations Results User benefits and improvements References Myoelectric-based hand gesture recognition for human-computer interface applications Abstract Introduction Background Current technologies and implementation References Video-based hand movement for human-computer interface Abstract Introduction Background Data analysis Discussion User requirements User benefits Shortcomings Future developments References Human-computer interface based on electrooculography Abstract Introduction Background Current technologies: Historical to state of the art Example of EOG-based system Results Discussion Limitations of the study Discussion: User benefits and limitations References Further reading Video-based eye tracking Abstract Introduction Background and history An example eye-tracking method Data analysis Results Discussion: User benefits and limitations References Speech for controlling computers Abstract Introduction History of speech-based machine commands Automatic speech recognition (ASR) Speech denoising methods Speech analysis fundamentals Subsections of speech: Phonemes How people speak: Speech production model Place principle hearing model Features selection for speech analysis Speech feature classification Artificial neural networks Limitations in current systems References Lip movement for human-computer interface Abstract Introduction: History and applications Current technologies User requirements Example of voiceless speech recognition systems Discussion: User benefits Summary References

[1]  Yskandar Hamam,et al.  Towards Inexpensive BCI Control for Wheelchair Navigation in the Enabled Environment - A Hardware Survey , 2010, Brain Informatics.

[2]  Kevin Englehart,et al.  High density electromyography data of normally limbed and transradial amputee subjects for multifunction prosthetic control. , 2012, Journal of electromyography and kinesiology : official journal of the International Society of Electrophysiological Kinesiology.

[3]  P.V.C. Hough,et al.  Machine Analysis of Bubble Chamber Pictures , 1959 .

[4]  Huosheng Hu,et al.  Myoelectric control systems - A survey , 2007, Biomed. Signal Process. Control..

[5]  Alireza Khotanzad,et al.  Invariant Image Recognition by Zernike Moments , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  P M Rossini,et al.  Mapping of motor cortical reorganization after stroke. A brain stimulation study with focal magnetic pulses. , 1997, Stroke.

[7]  Vuokko Lantz,et al.  Feasibility of building robust surface electromyography-based hand gesture interfaces , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[8]  Edward H. Adelson,et al.  Shiftable multiscale transforms , 1992, IEEE Trans. Inf. Theory.

[9]  Jerald D. Kralik,et al.  Real-time prediction of hand trajectory by ensembles of cortical neurons in primates , 2000, Nature.

[10]  Francesco Carlo Morabito,et al.  A Morlet wavelet classification technique for ICA filtered sEMG experimental data , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[11]  A. J. Fridlund,et al.  Guidelines for human electromyographic research. , 1986, Psychophysiology.

[12]  T. Jung,et al.  Assessing the feasibility of online SSVEP decoding in human walking using a consumer EEG headset , 2014, Journal of NeuroEngineering and Rehabilitation.

[13]  Nitish V. Thakor,et al.  Real-time myoelectric decoding of individual finger movements for a virtual target task , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[14]  James W. Davis,et al.  The Recognition of Human Movement Using Temporal Templates , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Terrence J. Sejnowski,et al.  The “independent components” of natural scenes are edge filters , 1997, Vision Research.

[16]  Mubarak Shah,et al.  Visual gesture recognition , 1994 .

[17]  C. A. Weaver,et al.  Psychology of Reading , 2012 .

[18]  Ching-Tien Shih,et al.  Assisting people with multiple disabilities to use computers with multiple mice. , 2009, Research in developmental disabilities.

[19]  G.C. Burdea,et al.  Virtual reality-enhanced stroke rehabilitation , 2001, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[20]  D.K. Kumar,et al.  Fractal Based Modelling and Analysis of Electromyography (EMG) To Identify Subtle Actions , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[21]  Dinesh Kumar,et al.  Classification of EOG for human computer interface , 2002, Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] [Engineering in Medicine and Biology.

[22]  Peter J. Kyberd,et al.  EMG signal filtering based on Empirical Mode Decomposition , 2006, Biomed. Signal Process. Control..

[23]  S Makeig,et al.  Spatially independent activity patterns in functional MRI data during the stroop color-naming task. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[24]  D. Regan Steady-state evoked potentials. , 1977, Journal of the Optical Society of America.

[25]  Keigo Hirakawa,et al.  Color constancy beyond bags of pixels , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  B. Hughes,et al.  ATP-dependent regulation of inwardly rectifying K+ current in bovine retinal pigment epithelial cells. , 1998, American journal of physiology. Cell physiology.

[27]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  N.V. Thakor,et al.  Towards the Control of Individual Fingers of a Prosthetic Hand Using Surface EMG Signals , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[29]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[30]  Stephen J. Roberts,et al.  Bayesian inference for an adaptive Ordered Probit model: An application to Brain Computer Interfacing , 2011, Neural Networks.

[31]  Terrence Fong,et al.  Novel interfaces for remote driving: gesture, haptic, and PDA , 2001, SPIE Optics East.

[32]  Clemens Brunner,et al.  BCI Software Platforms , 2012 .

[33]  I. Lesný,et al.  [Principles of electroencephalography]. , 1953, Prakticky lekar.

[34]  Piyush Kumar,et al.  Hand data glove: A new generation real-time mouse for Human-Computer Interaction , 2012, 2012 1st International Conference on Recent Advances in Information Technology (RAIT).

[35]  Anatole Lécuyer,et al.  Classifying EEG for brain computer interfaces using Gaussian processes , 2008, Pattern Recognit. Lett..

[36]  Yuttapong Rangsanseri,et al.  Wavelet-based texture analysis for SAR image classification , 1999, IEEE 1999 International Geoscience and Remote Sensing Symposium. IGARSS'99 (Cat. No.99CH36293).

[37]  Daniel Graupe,et al.  Functional Separation of EMG Signals via ARMA Identification Methods for Prosthesis Control Purposes , 1975, IEEE Transactions on Systems, Man, and Cybernetics.

[38]  L. Rizzolo,et al.  The distribution of Na+,K(+)-ATPase and 5A11 antigen in apical microvilli of the retinal pigment epithelium is unrelated to alpha-spectrin. , 1995, Journal of cell science.

[39]  David Zeltzer,et al.  A survey of glove-based input , 1994, IEEE Computer Graphics and Applications.

[40]  Thad Starner,et al.  Visual Recognition of American Sign Language Using Hidden Markov Models. , 1995 .

[41]  Aapo Hyvärinen,et al.  Fast and robust fixed-point algorithms for independent component analysis , 1999, IEEE Trans. Neural Networks.

[42]  L. Rizzolo Polarization of the Na+, K(+)-ATPase in epithelia derived from the neuroepithelium. , 1999, International review of cytology.

[43]  Arie E. Kaufman,et al.  An eye tracking computer user interface , 1993, Proceedings of 1993 IEEE Research Properties in Virtual Reality Symposium.

[44]  Joseph A. Paradiso,et al.  An Inertial Measurement Framework for Gesture Recognition and Applications , 2001, Gesture Workshop.

[45]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[46]  William Buxton,et al.  Issues and techniques in touch-sensitive tablet input , 1985, SIGGRAPH '85.

[47]  G. Pfurtscheller,et al.  Graz-BCI: state of the art and clinical applications , 2003, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[48]  S. Mallat A wavelet tour of signal processing , 1998 .

[49]  James W. Davis,et al.  Virtual PAT: A Virtual Personal Aerobics Trainer , 1998 .

[50]  Ming-Kuei Hu,et al.  Visual pattern recognition by moment invariants , 1962, IRE Trans. Inf. Theory.

[51]  P. Swerling Modern state estimation methods from the viewpoint of the method of least squares , 1971 .

[52]  R. S. Alexander Textbook of Medical Physiology, 5th edition. Arthur C. Guyton , 1977 .

[53]  Dezhong Yao,et al.  Stimulator selection in SSVEP-based BCI. , 2008, Medical engineering & physics.

[54]  Wu.P. Hope,et al.  Modular neural networks applied to surface EMG signals for limb function identification , 1999, IEEE SMC'99 Conference Proceedings. 1999 IEEE International Conference on Systems, Man, and Cybernetics (Cat. No.99CH37028).

[55]  Nikolaus Weiskopf,et al.  Real-time fMRI neurofeedback , 2016 .

[56]  George H Lawrence Biofeedback and Performance: An Update , 1984 .

[57]  B. Hughes,et al.  An outwardly rectifying K+ current active near resting potential in human retinal pigment epithelial cells. , 1995, The American journal of physiology.

[58]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[59]  Febo Cincotti,et al.  High-resolution EEG techniques for brain–computer interface applications , 2008, Journal of Neuroscience Methods.

[60]  Brendan Z. Allison,et al.  BCIs That Use Steady-State Visual Evoked Potentials or Slow Cortical Potentials , 2012 .

[61]  M. Marmor From sea lemons to c-waves , 1983, Cellular and Molecular Neurobiology.

[62]  I. Daubechies Orthonormal bases of compactly supported wavelets , 1988 .

[63]  Kevin Englehart,et al.  A multi-expert speech recognition system using acoustic and myoelectric signals , 2002, Proceedings of the Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society] [Engineering in Medicine and Biology.

[64]  Vijay P. Singh,et al.  Towards identification of finger flexions using single channel surface electromyography – able bodied and amputee subjects , 2013, Journal of NeuroEngineering and Rehabilitation.

[65]  Peter Olivieri,et al.  EagleEyes: An Eye Control System for Persons with Disabilities , 2013 .

[66]  Michel Beaudouin-Lafon,et al.  Charade: remote control of objects using free-hand gestures , 1993, CACM.

[67]  H. Devries MUSCLES ALIVE-THEIR FUNCTIONS REVEALED BY ELECTROMYOGRAPHY , 1976 .

[68]  M. Marmor Clinical electrophysiology of the retinal pigment epithelium , 2004, Documenta Ophthalmologica.

[69]  John R. Hershey,et al.  Fusing Array Microphone and Stereo Vision for Improved Computer Interfaces , 2005 .

[70]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[71]  A. Adjoudani,et al.  On the Integration of Auditory and Visual Parameters in an HMM-based ASR , 1996 .

[72]  Barak A. Pearlmutter,et al.  Blind Source Separation by Sparse Decomposition in a Signal Dictionary , 2001, Neural Computation.

[73]  Dinesh Kant Kumar,et al.  Evaluating Video and Facial Muscle Activity for a Better Assistive Technology: A Silent Speech Based HCI , 2014, Computational Models of Complex Systems.

[74]  Dinesh Kant Kumar,et al.  Decoding subtle forearm flexions using fractal features of surface electromyogram from single and multiple sensors , 2010, Journal of NeuroEngineering and Rehabilitation.

[75]  Kuntal Sengupta,et al.  Audio-visual modeling for bimodal speech recognition , 2001, 2001 IEEE International Conference on Systems, Man and Cybernetics. e-Systems and e-Man for Cybernetics in Cyberspace (Cat.No.01CH37236).

[76]  Hideyuki Sawada,et al.  Intuitive Manipulation of a Haptic Monitor for the Gestural Human-Computer Interaction , 2003, Gesture Workshop.

[77]  Milad Alemzadeh,et al.  Human-Computer Interaction: Overview on State of the Art , 2008 .

[78]  Geoffrey E. Hinton,et al.  Glove-Talk: a neural network interface between a data-glove and a speech synthesizer , 1993, IEEE Trans. Neural Networks.

[79]  Wai Chee Yau,et al.  Video analysis of mouth movement using motion templates for computer-based lip-reading , 2008 .

[80]  R. H. Steinberg Interactions between the retinal pigment epithelium and the neural retina , 1985, Documenta Ophthalmologica.

[81]  Linden J. Ball,et al.  Eye Tracking in Human-Computer Interaction and Usability Research : Current Status and Future Prospects , 2004 .

[82]  Roland T. Chin,et al.  On Image Analysis by the Methods of Moments , 1988, IEEE Trans. Pattern Anal. Mach. Intell..

[83]  Mats Hagberg,et al.  Computer mouse and track-ball operation:: Similarities and differences in posture, muscular load and perceived exertion , 1999 .

[84]  Marc M. Sebrechts,et al.  A gaze contingent environment for fostering social attention in autistic children , 2004, ETRA.

[85]  J.J. Vidal,et al.  Real-time detection of brain events in EEG , 1977, Proceedings of the IEEE.

[86]  Jochen Triesch,et al.  Robust classification of hand postures against complex backgrounds , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[87]  Tamás Roska,et al.  Face and Eye Detection by CNN Algorithms , 1999, J. VLSI Signal Process..

[88]  Dinesh Kumar,et al.  Motion Features for Visual Speech Recognition , 2009 .

[89]  Pierre Comon,et al.  Independent component analysis, A new concept? , 1994, Signal Process..

[90]  César Augusto Martins Pereira,et al.  Development and Evaluation of a Head-Controlled Human-Computer Interface with Mouse-Like Functions for Physically Disabled Users , 2009, Clinics.

[91]  James Church,et al.  Wearable sensor badge and sensor jacket for context awareness , 1999, Digest of Papers. Third International Symposium on Wearable Computers.

[92]  J. Little,et al.  Describing motion for recognition , 1995, Proceedings of International Symposium on Computer Vision - ISCV.

[93]  M. Bodruzzaman,et al.  Feature extraction using wavelet transform for neural network based image classification , 1998, Proceedings of Thirtieth Southeastern Symposium on System Theory.

[94]  Yoshiki Uchikawa,et al.  Development of eye pointer with free head-motion [EOG-based system] , 1998, Proceedings of the 20th Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Vol.20 Biomedical Engineering Towards the Year 2000 and Beyond (Cat. No.98CH36286).

[95]  David M. Santucci,et al.  Learning to Control a Brain–Machine Interface for Reaching and Grasping by Primates , 2003, PLoS biology.

[96]  Giulio Sandini,et al.  Fine detection of grasp force and posture by amputees via surface electromyography , 2009, Journal of Physiology-Paris.

[97]  Dinesh Kant Kumar,et al.  Thresholding Wavelet Networks for Signal Classification , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[98]  Sanjay Kumar,et al.  Visual Hand Gestures Classification Using Wavelet Transforms , 2003, Int. J. Wavelets Multiresolution Inf. Process..

[99]  Timothy J. Hazen Visual model structures and synchrony constraints for audio-visual speech recognition , 2006, IEEE Transactions on Audio, Speech, and Language Processing.

[100]  Thierry Dutoit,et al.  Performance of the Emotiv Epoc headset for P300-based applications , 2013, Biomedical engineering online.

[101]  Sunghoon Kwon,et al.  EOG-based glasses-type wireless mouse for the disabled , 1999, Proceedings of the First Joint BMES/EMBS Conference. 1999 IEEE Engineering in Medicine and Biology 21st Annual Conference and the 1999 Annual Fall Meeting of the Biomedical Engineering Society (Cat. N.

[102]  Manuel Mazo,et al.  EOG guidance of a wheelchair using neural networks , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[103]  F. K. Lam,et al.  Fuzzy EMG classification for prosthesis control. , 2000, IEEE transactions on rehabilitation engineering : a publication of the IEEE Engineering in Medicine and Biology Society.

[104]  Ivan Laptev,et al.  Tracking of Multi-state Hand Models Using Particle Filtering and a Hierarchy of Multi-scale Image Features , 2001, Scale-Space.

[105]  Holly A. Yanco,et al.  Wheelesley: A Robotic Wheelchair System: Indoor Navigation and User Interface , 1998, Assistive Technology and Artificial Intelligence.

[106]  Nitish V. Thakor,et al.  Decoding of Individuated Finger Movements Using Surface Electromyography , 2009, IEEE Transactions on Biomedical Engineering.

[107]  Michael Bach,et al.  ISCEV standard for clinical electro-oculography (2010 update) , 2011, Documenta ophthalmologica. Advances in ophthalmology.

[108]  Todd Kuiken,et al.  Pattern recognition control of multifunction myoelectric prostheses by patients with congenital transradial limb defects: a preliminary study , 2011, Prosthetics and orthotics international.

[109]  Lars Bretzner,et al.  A Prototype System for Computer Vision Based Human Computer Interaction , 2001 .

[110]  Sridhar P. Arjunan,et al.  FRACTAL PROPERTIES OF SURFACE ELECTROMYOGRAM FOR CLASSIFICATION OF LOW-LEVEL HAND MOVEMENTS FROM SINGLE-CHANNEL FOREARM MUSCLE ACTIVITY , 2011 .

[111]  J J Vidal,et al.  Toward direct brain-computer communication. , 1973, Annual review of biophysics and bioengineering.

[112]  Rory A. Cooper,et al.  Intelligent control of power wheelchairs , 1995 .

[113]  Te-Won Lee,et al.  Blind Separation of Delayed and Convolved Sources , 1996, NIPS.

[114]  Eric D. Petajan Automatic lipreading to enhance speech recognition , 1984 .

[115]  Ye Zhang,et al.  The Head-trace Mouse for Elderly : A Human-Computer Interaction System Based on Detection of Poses of Head and Mouth , 2013 .

[116]  Pornchai Phukpattaranont,et al.  Fractal analysis features for weak and single-channel upper-limb EMG signals , 2012, Expert Syst. Appl..

[117]  Milyn C. Moy Gesture-Based Interaction with a Pet Robot , 1999, AAAI/IAAI.

[118]  M. Teague Image analysis via the general theory of moments , 1980 .

[119]  G. Sperling,et al.  Attentional modulation of SSVEP power depends on the network tagged by the flicker frequency. , 2006, Cerebral cortex.

[120]  Reshma Khemchandani,et al.  Twin Support Vector Machines for Pattern Classification , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[121]  P. Nunez,et al.  Steady state visually evoked potential (SSVEP) topography in a graded working memory task. , 2001, International journal of psychophysiology : official journal of the International Organization of Psychophysiology.

[122]  Kevin B. Englehart,et al.  A wavelet-based continuous classification scheme for multifunction myoelectric control , 2001, IEEE Transactions on Biomedical Engineering.

[123]  Noureddine Cherabit,et al.  Circular Hough Transform for Iris localization , 2012 .

[124]  D. Kumar,et al.  Representation and Classification of Human Movement Using Temporal Templates and Statistical Measure of Similarity , 2022 .

[125]  S. Kumar,et al.  EMG based voice recognition , 2004, Proceedings of the 2004 Intelligent Sensors, Sensor Networks and Information Processing Conference, 2004..

[126]  Ning Ma,et al.  Classification of hand direction using multi-channel electromyography by neural network , 2001, The Seventh Australian and New Zealand Intelligent Information Systems Conference, 2001.

[127]  Hongying Meng,et al.  Singularity characteristics of needle EMG IP signals , 2006, IEEE Transactions on Biomedical Engineering.

[128]  李光林,et al.  Real-Time Performance of Textile Electrodes in Electromyogram Pattern-Recognition Based Prosthesis Control , 2012 .

[129]  F. L. D. Silva,et al.  Event-related EEG/MEG synchronization and desynchronization: basic principles , 1999, Clinical Neurophysiology.

[130]  Anupam Agrawal,et al.  Vision based hand gesture recognition for human computer interaction: a survey , 2012, Artificial Intelligence Review.

[131]  Jie Liu,et al.  SpeakerSense: Energy Efficient Unobtrusive Speaker Identification on Mobile Phones , 2011, Pervasive.

[132]  Chalapathy Neti,et al.  Recent advances in the automatic recognition of audiovisual speech , 2003, Proc. IEEE.

[133]  P. Kennedy,et al.  Restoration of neural output from a paralyzed patient by a direct brain connection , 1998, Neuroreport.

[134]  Mubarak Shah,et al.  Determining driver visual attention with one camera , 2003, IEEE Trans. Intell. Transp. Syst..

[135]  T. Kuiken,et al.  Quantifying Pattern Recognition—Based Myoelectric Control of Multifunctional Transradial Prostheses , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[136]  R. Schülke [Anatomy and physiology]. , 1968, Zahntechnik; Zeitschrift fur Theorie und Praxis der wissenschaftlichen Zahntechnik.

[137]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[138]  R.N. Scott,et al.  A new strategy for multifunction myoelectric control , 1993, IEEE Transactions on Biomedical Engineering.

[139]  T. Yagi,et al.  Biological interaction between man and machine , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[140]  Damjan Zazula,et al.  Decomposition of surface EMG signals using non-linear LMS optimisation of higher-order cumulants , 2002, Proceedings of 15th IEEE Symposium on Computer-Based Medical Systems (CBMS 2002).

[141]  Caroline Hummels,et al.  An Intuitive Two-Handed Gestural Interface for Computer Supported Product Design , 1997, Gesture Workshop.

[142]  Tommy Strandvall,et al.  Eye Tracking in Human-Computer Interaction and Usability Research , 2009, INTERACT.

[143]  Anthony J. Hornof,et al.  Eyedraw: a system for drawing pictures with eye movements , 2004, ACM SIGACCESS Access. Comput..

[144]  Iain A. Anderson,et al.  Stretch sensors for human body motion , 2014, Smart Structures.

[145]  Horst-Michael Groß,et al.  Neural Architecture for Gesture-Based Human-Machine-Interaction , 1997, Gesture Workshop.

[146]  Terrence J. Sejnowski,et al.  An Information-Maximization Approach to Blind Separation and Blind Deconvolution , 1995, Neural Computation.

[147]  S. Eckstein The Belmont Report: ethical principles and guidelines for the protection of human subjects of research , 2003 .