User-driven Intelligent Interface on the Basis of Multimodal Augmented Reality and Brain-Computer Interaction for People with Functional Disabilities

The analysis of the current integration attempts of some modes and use cases of human-to-machine interaction is presented. The new concept of the user-driven intelligent interface for accessibility is proposed on the basis of multimodal augmented reality and brain-computer interaction for various applications: in disabilities studies, education, home care, health care, eHealth, etc. The several use cases of multimodal augmentation are presented. The perspectives of the better human comprehension by the immediate feedback through neurophysical channels by means of brain-computer interaction are outlined. It is shown that brain–computer interface (BCI) technology provides new strategies to overcome limits of the currently available user interfaces, especially for people with functional disabilities. The results of the previous studies of the low end consumer and open-source BCI-devices allow us to conclude that combination of machine learning (ML), multimodal interactions (visual, sound, tactile) with BCI will profit from the immediate feedback from the actual neurophysical reactions classified by ML methods. In general, BCI in combination with other modes of AR interaction can deliver much more information than these types of interaction themselves. Even in the current state the combined AR-BCI interfaces could provide the highly adaptable and personal services, especially for people with functional disabilities.

[1]  Ivan Kastelan,et al.  E2LP: A Unified Embedded Engineering Learning Platform , 2013, 2013 Euromicro Conference on Digital System Design.

[2]  Yuri Gordienko,et al.  Deep Learning for Fatigue Estimation on the Basis of Multimodal Human-Machine Interactions , 2017, ArXiv.

[3]  K. Skala,et al.  Augmented Coaching Ecosystem for Non-obtrusive Adaptive Personalized Elderly Care on the basis of Cloud-Fog-Dew computing paradigm , 2017, 2017 40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[4]  Yuri Gordienko,et al.  Automatized generation of alphabets of symbols , 2017, 2017 Federated Conference on Computer Science and Information Systems (FedCSIS).

[5]  Cain C T Clark,et al.  A Review of Emerging Analytical Techniques for Objective Physical Activity Measurement in Humans , 2016, Sports Medicine.

[6]  Yuri Gordienko,et al.  Generating and Estimating Nonverbal Alphabets for Situated and Multimodal Communications , 2017, ArXiv.

[7]  José del R. Millán,et al.  Brain-Computer Interfaces , 2020, Handbook of Clinical Neurology.

[8]  Hee-Cheol Kim,et al.  A Review of Accelerometer-Based Physical Activity Measurement , 2011, ICITCS.

[9]  George Demiris,et al.  Understanding views on everyday use of personal health information: Insights from community dwelling older adults , 2018, Informatics for health & social care.

[10]  Martina Ziefle,et al.  Medical Technology in Smart Homes: Exploring the User's Perspective on Privacy, Intimacy and Trust , 2011, 2011 IEEE 35th Annual Computer Software and Applications Conference Workshops.

[11]  Nizar Bouhlel,et al.  New tools for automating tactile geographic map translation , 2014, ASSETS.

[12]  Rostislava Dimitrova Growth in the intersection of eHealth and active and healthy ageing. , 2013, Technology and health care : official journal of the European Society for Engineering and Medicine.

[13]  Guillaume Barrois,et al.  Dual-mode registration of dynamic contrast-enhanced ultrasound combining tissue and contrast sequences. , 2014, Ultrasonics.

[14]  Yuri Gordienko,et al.  Performance Analysis of Open Source Machine Learning Frameworks for Various Parameters in Single-Threaded and Multi-Threaded Modes , 2017, ArXiv.

[15]  Gilles Fedak,et al.  Synergy of volunteer measurements and volunteer computing for effective data collecting, processing, simulating and analyzing on a worldwide scale , 2015, 2015 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[16]  Nikita Gordienko Multi-Parametric Statistical Method for Estimation of Accumulated Fatigue by Sensors in Ordinary Gadgets , 2016, ArXiv.

[17]  Dennis J. McFarland,et al.  Brain–computer interfaces for communication and control , 2002, Clinical Neurophysiology.

[18]  Mark Billinghurst,et al.  A Survey of Augmented Reality , 2015, Found. Trends Hum. Comput. Interact..

[19]  Woodrow Barfield,et al.  Fundamentals of Wearable Computers and Augumented Reality , 2000 .

[20]  Ernst Fernando Lopes Da Silva Niedermeyer,et al.  Electroencephalography, basic principles, clinical applications, and related fields , 1982 .

[21]  Lennart E. Nacke,et al.  An Introduction to Physiological Player Metrics for Evaluating Games , 2013, Game Analytics, Maximizing the Value of Player Data.

[22]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[23]  Anabel Martin-Gonzalez,et al.  Intelligent Computing Systems , 2016, Communications in Computer and Information Science.

[24]  Mohd Azlishah Othman,et al.  Advances in Machine Learning and Signal Processing: Proceedings of MALSIP 2015 , 2016 .

[25]  Florent Souvestre,et al.  Augmented Reality Interface for E2LP: Assistance in Electronic Laboratories Through Augmented Reality , 2016 .