Toward Dynamically Adaptive Simulation: Multimodal Classification of User Expertise Using Wearable Devices

Simulation-based training has been proven to be a highly effective pedagogical strategy. However, misalignment between the participant’s level of expertise and the difficulty of the simulation has been shown to have significant negative impact on learning outcomes. To ensure that learning outcomes are achieved, we propose a novel framework for adaptive simulation with the goal of identifying the level of expertise of the learner, and dynamically modulating the simulation complexity to match the learner’s capability. To facilitate the development of this framework, we investigate the classification of expertise using biological signals monitored through wearable sensors. Trauma simulations were developed in which electrocardiogram (ECG) and galvanic skin response (GSR) signals of both novice and expert trauma responders were collected. These signals were then utilized to classify the responders’ expertise, successive to feature extraction and selection, using a number of machine learning methods. The results show the feasibility of utilizing these bio-signals for multimodal expertise classification to be used in adaptive simulation applications.

[1]  J. Sweller COGNITIVE LOAD THEORY, LEARNING DIFFICULTY, AND INSTRUCTIONAL DESIGN , 1994 .

[2]  Slava Kalyuga Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction , 2007 .

[3]  L. Breiman Random Forests--random Features , 1999 .

[4]  S. Issenberg,et al.  Simulation in healthcare education: A best evidence practical guide. AMEE Guide No. 82 , 2013, Medical teacher.

[5]  Kilseop Ryu,et al.  Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic , 2005 .

[6]  Bryan Reimer,et al.  Sensitivity of Physiological Measures for Detecting Systematic Variations in Cognitive Demand From a Working Memory Task , 2012, Hum. Factors.

[7]  Kyle Ross,et al.  Classification of Cognitive Load and Expertise for Adaptive Simulation using Deep Multitask Learning , 2019, 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII).

[8]  Peter E. Hart,et al.  Nearest neighbor pattern classification , 1967, IEEE Trans. Inf. Theory.

[9]  Slava Kalyuga,et al.  The Expertise Reversal Effect , 2003 .

[10]  Kyriaki Kalimeri,et al.  Cognitive Load Assessment from EEG and Peripheral Biosignals for the Design of Visually Impaired Mobility Aids , 2018, Wirel. Commun. Mob. Comput..

[11]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[12]  Jennifer Healey,et al.  Detecting stress during real-world driving tasks using physiological sensors , 2005, IEEE Transactions on Intelligent Transportation Systems.

[13]  Hans Rystedt,et al.  Realism, authenticity, and learning in healthcare simulations: rules of relevance and irrelevance as interactive achievements , 2012 .

[14]  R. Reznick,et al.  Training and simulation for patient safety , 2010, Quality and Safety in Health Care.

[15]  N. Lomb Least-squares frequency analysis of unequally spaced data , 1976 .

[16]  Philip H. Swain,et al.  Purdue e-Pubs , 2022 .

[17]  D. Hatzinakos,et al.  ECG Biometric Recognition Without Fiducial Detection , 2006, 2006 Biometrics Symposium: Special Session on Research at the Biometric Consortium Conference.

[18]  Huan Liu,et al.  Feature Selection for Classification: A Review , 2014, Data Classification: Algorithms and Applications.

[19]  G. Breithardt,et al.  Heart rate variability: standards of measurement, physiological interpretation and clinical use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. , 1996 .

[20]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[21]  Steffen Walter,et al.  Preliminary classification of cognitive load states in a human machine interaction scenario , 2017, 2017 International Conference on Companion Technology (ICCT).

[22]  Susan Galloway,et al.  Simulation Techniques to Bridge the Gap Between Novice and Competent Healthcare Professionals , 2009, OJIN: The Online Journal of Issues in Nursing.

[23]  John Sweller,et al.  Cognitive Load Theory for the Design of Medical Simulations , 2015, Simulation in healthcare : journal of the Society for Simulation in Healthcare.

[24]  M. Good,et al.  Patient simulation for training basic and advanced clinical skills , 2003, Medical education.

[25]  F. Paas,et al.  Measurement of Cognitive Load in Instructional Research , 1994, Perceptual and motor skills.

[26]  Rafael A. Calvo,et al.  Effect of Experimental Factors on the Recognition of Affective Mental States through Physiological Measures , 2009, Australasian Conference on Artificial Intelligence.

[27]  Yang Wang,et al.  The Effect of Stress on Cognitive Load Measurement , 2013, INTERACT.

[28]  S. Ali Etemad,et al.  Dynamically adaptive simulation based on expertise and cognitive load , 2018, 2018 IEEE Games, Entertainment, Media Conference (GEM).

[29]  Willis J. Tompkins,et al.  A Real-Time QRS Detection Algorithm , 1985, IEEE Transactions on Biomedical Engineering.

[30]  J. Sweller,et al.  Cognitive load theory in health professional education: design principles and strategies , 2010, Medical education.

[31]  John Sweller,et al.  Cognitive Load During Problem Solving: Effects on Learning , 1988, Cogn. Sci..

[32]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[33]  Vimla L. Patel,et al.  The multitasking clinician: Decision-making and cognitive demand during and after team handoffs in emergency care , 2007, Int. J. Medical Informatics.

[34]  P. S. Hiremath,et al.  Performance Comparison Of Ann Classifiers For Sleep Apnea Detection Based On Ecg Signal Analysis Using Hilbert Transform , 2018, INTERNATIONAL JOURNAL OF COMPUTERS & TECHNOLOGY.