Towards the Optimization of Gaze-Controlled Human-Computer Interaction: Application to Hindi Virtual Keyboard for Stroke Patients

Virtual keyboard applications and alternative communication devices provide new means of communication to assist disabled people. To date, virtual keyboard optimization schemes based on script-specific information, along with multimodal input access facility, are limited. In this paper, we propose a novel method for optimizing the position of the displayed items for gaze-controlled tree-based menu selection systems by considering a combination of letter frequency and command selection time. The optimized graphical user interface layout has been designed for a Hindi language virtual keyboard based on a menu wherein 10 commands provide access to type 88 different characters, along with additional text editing commands. The system can be controlled in two different modes: eye-tracking alone and eye-tracking with an access soft-switch. Five different keyboard layouts have been presented and evaluated with ten healthy participants. Furthermore, the two best performing keyboard layouts have been evaluated with eye-tracking alone on ten stroke patients. The overall performance analysis demonstrated significantly superior typing performance, high usability (87% SUS score), and low workload (NASA TLX with 17 scores) for the letter frequency and time-based organization with script specific arrangement design. This paper represents the first optimized gaze-controlled Hindi virtual keyboard, which can be extended to other languages.

[1]  H. Hodkinson.  Evaluation of a mental test score for assessment of mental impairment in the elderly. , 1972, Age and ageing.

[2]  R. Lyle.  A performance test for assessment of upper limb function in physical rehabilitation treatment and research , 1981, International journal of rehabilitation research. Internationale Zeitschrift fur Rehabilitationsforschung. Revue internationale de recherches de readaptation.

[3]  Robert J. K. Jacob,et al.  What you look at is what you get: eye movement-based interaction techniques , 1990, CHI '90.

[4]  T. Mohanan.  Argument structure in Hindi , 1994 .

[5]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[6]  Sharon Glennen,et al.  The handbook of augmentative and alternative communication , 1997 .

[7]  Durgesh Rao,et al.  An intelligent multi-layered input scheme for phonetic scripts , 2002, SMARTGRAPH '02.

[8]  David J. Ward,et al.  Artificial intelligence: Fast hands-free writing by gaze direction , 2002, Nature.

[9]  Anirudha Joshi,et al.  Keylekh: a keyboard for text entry in indic scripts , 2004, CHI EA '04.

[10]  Denis Anson,et al.  The Effects of Word Completion and Word Prediction on Typing Rates Using On-Screen Keyboards , 2006, Assistive technology : the official journal of RESNA.

[11]  K. Wilkinson,et al.  The state of research and practice in augmentative and alternative communication for children with developmental/intellectual disabilities. , 2007, Mental retardation and developmental disabilities research reviews.

[12]  J. R. Wolpaw,et al.  Brain–computer interfaces (BCIs): Detection instead of classification , 2008, Journal of Neuroscience Methods.

[13]  D. Samanta,et al.  Friend: A Communication Aid for Persons With Disabilities , 2008, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[14]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[15]  Cheng-Seen Ho,et al.  A Fast Text-Based Communication System for Handicapped Aphasiacs , 2009, 2009 Fifth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[16]  Octavio Rivera,et al.  Measuring Performance of Virtual Keyboards Based on Cyclic Scanning , 2009, 2009 Fifth International Conference on Autonomic and Autonomous Systems.

[17]  S. Ghosh,et al.  Virtual keyboard design: State of the arts and research issues , 2010, 2010 IEEE Students Technology Symposium (TechSym).

[18]  Serkan Gurkan,et al.  Design of a Novel Efficient Human–Computer Interface: An Electrooculagram Based Virtual Keyboard , 2010, IEEE Transactions on Instrumentation and Measurement.

[19]  J. Wolpaw,et al.  A novel P300-based brain–computer interface stimulus presentation paradigm: Moving beyond rows and columns , 2010, Clinical Neurophysiology.

[20]  Hubert Cecotti,et al.  A Self-Paced and Calibration-Less SSVEP-Based Brain–Computer Interface Speller , 2010, IEEE Transactions on Neural Systems and Rehabilitation Engineering.

[21]  Giuseppe Riva,et al.  The combined use of Brain Computer Interface and Eye-Tracking technology for cognitive assessment in Amyotrophic Lateral Sclerosis , 2011, 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[22]  Soumalya Ghosh,et al.  Designing an efficient virtual keyboard for text composition in Bengali , 2011, IndiaHCI.

[23]  Girijesh Prasad,et al.  Designing a virtual keyboard with multi-modal access for people with disabilities , 2011, 2011 World Congress on Information and Communication Technologies.

[24]  M. L. Mele,et al.  A psychotechnological review on eye-tracking systems: towards user experience , 2012, Disability and rehabilitation. Assistive technology.

[25]  Si-Woon Park,et al.  Augmentative and Alternative Communication Training Using Eye Blink Switch for Locked-in Syndrome Patient , 2012, Annals of rehabilitation medicine.

[26]  Samit Bhattacharya,et al.  Bengali text input interface design for mobile devices , 2012, Universal Access in the Information Society.

[27]  Brian Roark,et al.  RSVP keyboard: An EEG based typing interface , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[28]  Soumalya Ghosh,et al.  An Approach to Design Virtual Keyboards for Text Composition in Indian Languages , 2013, Int. J. Hum. Comput. Interact..

[29]  Sayan Sarcar,et al.  Eyeboard++: an enhanced eye gaze-based text entry system in Hindi , 2013, APCHI.

[30]  Jean-Yves Antoine,et al.  Effect of dynamic keyboard and word-prediction systems on text input speed in persons with functional tetraplegia. , 2014, Journal of rehabilitation research and development.

[31]  Edwin S. Dalmaijer,et al.  Is the low-cost EyeTribe eye tracker any good for research? , 2014 .

[32]  Per Ola Kristensson,et al.  An evaluation of Dasher with a high-performance language model as a gaze communication method , 2014, AVI.

[33]  Girijesh Prasad,et al.  Enhancing an Eye-Tracker based Human-Computer Interface with Multi-modal Accessibility Applied for Text Entry , 2015 .

[34]  Anupam Basu,et al.  Design and evaluation of Unicode compliance Meitei/Meetei Mayek keyboard layout , 2015, 2015 International Symposium on Advanced Computing and Communication (ISACC).

[35]  KongFatt Wong-Lin,et al.  Towards increasing the number of commands in a hybrid brain-computer interface with combination of gaze and motor imagery , 2015, 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[36]  Hubert Cecotti,et al.  A Multimodal Gaze-Controlled Virtual Keyboard , 2016, IEEE Transactions on Human-Machine Systems.

[37]  KongFatt Wong-Lin,et al.  A novel multimodal gaze-controlled Hindi virtual keyboard for disabled users , 2016, 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[38]  Edna Lúcia Flôres,et al.  A new concept of assistive virtual keyboards based on a systematic review of text entry optimization techniques , 2016 .

[39]  Anirban Dutta,et al.  SmartEye: Developing a Novel Eye Tracking System for Quantitative Assessment of Oculomotor Abnormalities , 2016, IEEE Transactions on Neural Systems and Rehabilitation Engineering.