A Mobile-Based Multimodal Framework for Pervasive Health Monitoring

Pervasive health monitoring tools and applications based on mobile devices can be used for digital phenotyping, thereby facilitating diagnosis and treatment of diseases. We present the development of an unobtrusive health monitoring framework, enabling multimodal data collection of activity information from wearable devices, smartphones, and third-party apps, keystroke typing data, location information, data from medical and environmental IoT devices, and self-reports. The architectural elements of the framework are presented, and proof-of-concept case studies based on the developed framework are demonstrated, to show its feasibility and value in building different pervasive health applications.

[1]  G. Clifford,et al.  Daily longitudinal self-monitoring of mood variability in bipolar disorder and borderline personality disorder , 2016, Journal of affective disorders.

[2]  M. Keshavan,et al.  COVID-19, mobile health and serious mental illness , 2020, Schizophrenia Research.

[3]  W. Marsden.  I and J , 2012 .

[4]  Emiliano Miluzzo,et al.  A survey of mobile phone sensing , 2010, IEEE Communications Magazine.

[5]  Vera Maljkovic,et al.  Developing Measures of Cognitive Impairment in the Real World from Consumer-Grade Multimodal Sensor Streams , 2019, KDD.

[6]  Mirco Musolesi,et al.  Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis , 2015, UbiComp.

[7]  S. Rauch,et al.  Harnessing Smartphone-Based Digital Phenotyping to Enhance Behavioral and Mental Health , 2016, Neuropsychopharmacology.

[8]  Michael Riegler,et al.  Mental health monitoring with multimodal sensing and machine learning: A survey , 2018, Pervasive Mob. Comput..

[9]  Dimitrios Iakovakis,et al.  Touchscreen typing-pattern analysis for detecting fine motor skills decline in early-stage Parkinson’s disease , 2018, Scientific reports.

[10]  L. Giancardo,et al.  Psychomotor Impairment Detection via Finger Interactions with a Computer Keyboard During Natural Typing , 2015, Scientific Reports.

[11]  J Quinn.  An HL7 (Health Level Seven) overview. , 1999, Journal of AHIMA.

[12]  S. Gosling,et al.  Do people know how they behave? Self-reported act frequencies compared with on-line codings by observers. , 1998, Journal of personality and social psychology.

[13]  Sofia B. Dias,et al.  Motor Impairment Estimates via Touchscreen Typing Dynamics Toward Parkinson's Disease Detection From Data Harvested In-the-Wild , 2018, Front. ICT.

[14]  J. D. Runyan,et al.  A Smartphone Ecological Momentary Assessment/Intervention “App” for Collecting Real-Time Data and Promoting Self-Awareness , 2013, PloS one.

[15]  Ingemar J. Cox,et al.  Digital technologies in the public-health response to COVID-19 , 2020, Nature Medicine.

[16]  Dimitrios Iakovakis,et al.  Touchscreen typing pattern analysis for remote detection of the depressive tendency , 2019, Scientific Reports.

[17]  D. Bogen,et al.  Feasibility and acceptability of a mobile app in an ecological momentary assessment of early breastfeeding , 2017, Maternal & child nutrition.

[18]  P. Dagum,et al.  On the Need for Digital Phenotyping to Obtain Insights into Mental States in the COVID-19 Pandemic , 2020, Digital Psychology.

[19]  John Torous,et al.  Creating a Digital Health Smartphone App and Digital Phenotyping Platform for Mental Health and Diverse Healthcare Needs: an Interdisciplinary and Collaborative Approach , 2019, Journal of Technology in Behavioral Science.

[20]  M. McInnis,et al.  Predicting Mood Disturbance Severity with Mobile Phone Keystroke Metadata: A BiAffect Digital Phenotyping Study , 2018, Journal of medical Internet research.

[21]  Vassilis Koutkias,et al.  A Survey of Mobile Phone Sensing, Self-Reporting, and Social Sharing for Pervasive Healthcare , 2017, IEEE Journal of Biomedical and Health Informatics.

[22]  Sam Ruby,et al.  RESTful Web Services , 2007 .

[23]  Dario Salvi,et al.  Mobistudy: An Open Mobile-Health Platform for Clinical Research , 2019, 2019 IEEE 19th International Conference on Bioinformatics and Bioengineering (BIBE).

[24]  Jadwiga Indulska,et al.  Advancing Android activity recognition service with Markov smoother: Practical solutions , 2017, Pervasive Mob. Comput..