AMSER: Adaptive Multimodal Sensing for Energy Efficient and Resilient eHealth Systems

eHealth systems deliver critical digital healthcare and wellness services for users by continuously monitoring physiological and contextual data. eHealth applications use multimodal machine learning kernels to analyze data from different sensor modalities and automate decision-making. Noisy inputs and motion artifacts during sensory data acquisition affect the i) prediction accuracy and resilience of eHealth services and ii) energy efficiency in processing garbage data. Monitoring raw sensory inputs to identify and drop data and features from noisy modalities can improve prediction accuracy and energy efficiency. We propose a closed-loop monitoring and control framework for multi-modal eHealth applications, AMSER, that can mitigate garbage-in garbage-out by i) monitoring input modalities, ii) analyzing raw input to selectively drop noisy data and features, and iii) choosing appropriate machine learning models that fit the configured data and feature vector to improve prediction accuracy and energy efficiency. We evaluate our AMSER approach using multi-modal eHealth applications of pain assessment and stress monitoring over different levels and types of noisy components incurred via different sensor modalities. Our approach achieves up to 22% improvement in prediction accuracy and 5.6× energy consumption reduction in the sensing phase against the state-of-the-art multi-modal monitoring application.

[1]  Arkady B. Zaslavsky,et al.  Context Aware Computing for The Internet of Things: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[2]  Emad Kasaeyan Naeini,et al.  Objective Pain Assessment Using Wrist-based PPG Signals: A Respiratory Rate Based Method , 2021, 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[3]  Qiang Chen,et al.  A Health-IoT Platform Based on the Integration of Intelligent Packaging, Unobtrusive Bio-Sensor, and Intelligent Medicine Box , 2014, IEEE Transactions on Industrial Informatics.

[4]  Fakhri Karray,et al.  A comprehensive survey on multimodal medical signals fusion for smart healthcare systems , 2021, Inf. Fusion.

[5]  Patrick Thiam,et al.  Multimodal Data Fusion for Person-Independent, Continuous Estimation of Pain Intensity , 2015, EANN.

[6]  Kristof Van Laerhoven,et al.  Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection , 2018, ICMI.

[7]  Pasi Liljeberg,et al.  A Real-time PPG Quality Assessment Approach for Healthcare Internet-of-Things , 2019, ANT/EDI40.

[8]  R. B. Deshmukh,et al.  A Systematic Review of Compressive Sensing: Concepts, Implementations and Applications , 2018, IEEE Access.

[9]  Rosalind W. Picard,et al.  Automatic Recognition Methods Supporting Pain Assessment: A Survey , 2019, IEEE Transactions on Affective Computing.

[10]  Marco Levorato,et al.  Personalized Stress Monitoring using Wearable Sensors in Everyday Settings , 2021, 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC).

[11]  David Atienza,et al.  Self-Aware Machine Learning for Multimodal Workload Monitoring during Manual Labor on Edge Wearable Sensors , 2020, IEEE Design & Test.

[12]  Jon Nicholl,et al.  Role of ambulance response times in the survival of patients with out-of-hospital cardiac arrest , 2010, Emergency Medicine Journal.

[13]  Ki H. Chon,et al.  Electrodermal Activity Is Sensitive to Cognitive Stress under Water , 2018, Front. Physiol..

[14]  N. Dutt,et al.  Prospective Study Evaluating a Pain Assessment Tool in a Postoperative Environment: Protocol for Algorithm Testing and Enhancement , 2020, JMIR research protocols.

[15]  Nikil Dutt,et al.  An Edge-Assisted and Smart System for Real-Time Pain Monitoring , 2019, 2019 IEEE/ACM International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE).

[16]  David Atienza,et al.  Multi-Modal Acute Stress Recognition Using Off-the-Shelf Wearable Devices , 2019, 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[17]  Hui Liu,et al.  On-Demand Deep Model Compression for Mobile Devices: A Usage-Driven Model Selection Framework , 2018, MobiSys.

[18]  N. Dutt,et al.  Pain Assessment Tool With Electrodermal Activity for Postoperative Patients: Method Validation Study , 2021, JMIR mHealth and uHealth.

[20]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[21]  Marco Levorato,et al.  Context-Aware Sensing via Dynamic Programming for Edge-Assisted Wearable Systems , 2020, ACM Trans. Comput. Heal..

[22]  Sharief M. A. Oteafy A Framework for Heterogeneous Sensing in Big Sensed Data , 2016, 2016 IEEE Global Communications Conference (GLOBECOM).

[23]  M. Benedek,et al.  A continuous measure of phasic electrodermal activity , 2010, Journal of Neuroscience Methods.

[24]  Nikil D. Dutt,et al.  pyEDA: An Open-Source Python Toolkit for Pre-processing and Feature Extraction of Electrodermal Activity , 2021, ANT/EDI40.

[25]  P. Liljeberg,et al.  Pain Recognition With Electrocardiographic Features in Postoperative Patients: Method Validation Study , 2021, Journal of medical Internet research.