A Deep Learning Approach for Recognizing Activity of Daily Living (ADL) for Senior Care: Exploiting Interaction Dependency and Temporal Patterns

Ensuring the health and safety of senior citizens who live alone is a growing societal concern. The Activity of Daily Living (ADL) approach is a common means to monitor disease progression and the ability of these individuals to care for themselves. However, the prevailing sensor-based ADL monitoring systems primarily rely on wearable motion sensors, capture insufficient information for accurate ADL recognition, and do not provide a comprehensive understanding of ADLs at different granularities. Current healthcare IS and mobile analytics research focuses on studying the system, device, and provided services, and is in need of an endto-end solution to comprehensively recognize ADLs based on mobile sensor data. This study adopts the design science paradigm and employs advanced deep learning algorithms to develop a novel hierarchical, multiphase ADL recognition framework to model ADLs at different granularities. We propose a novel 2D interaction kernel for convolutional neural networks to leverage interactions between human and object motion sensors. We rigorously evaluate each proposed module and the entire framework against state-ofthe-art benchmarks (e.g., support vector machines, DeepConvLSTM, hidden Markov models, and topicmodeling-based ADLR) on two real-life motion sensor datasets that consist of ADLs at varying granularities: Opportunity and INTER. Results and a case study demonstrate that our framework can recognize ADLs at different levels more accurately. We discuss how stakeholders can further benefit from our proposed framework. Beyond demonstrating practical utility, we discuss contributions to the IS knowledge base for future design science-based cybersecurity, healthcare, and mobile analytics applications.

[1]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[2]  Guang-Zhong Yang,et al.  Sensor Positioning for Activity Recognition Using Wearable Accelerometers , 2011, IEEE Transactions on Biomedical Circuits and Systems.

[3]  Roberto Turrin,et al.  Performance of recommender algorithms on top-n recommendation tasks , 2010, RecSys '10.

[4]  Ritu Agarwal,et al.  The Digitization of Healthcare: Boundary Risks, Emotion, and Consumer Willingness to Disclose Personal Health Information , 2011, Inf. Syst. Res..

[5]  Sydney Katz Assessing Self‐maintenance: Activities of Daily Living, Mobility, and Instrumental Activities of Daily Living , 1983, Journal of the American Geriatrics Society.

[6]  George Forman,et al.  An Extensive Empirical Study of Feature Selection Metrics for Text Classification , 2003, J. Mach. Learn. Res..

[7]  Xiao-Bai Li,et al.  Anonymizing and Sharing Medical Text Records , 2017, Inf. Syst. Res..

[8]  Zhiqiang Zheng,et al.  The Impact of Health Information Sharing on Duplicate Testing , 2017, MIS Q..

[9]  Hsinchun Chen,et al.  SilverLink: Developing an International Smart and Connected Home Monitoring System for Senior Care , 2016, ICSH.

[10]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[11]  Arun Rai,et al.  Editor's comments: diversity of Design Science Research , 2017 .

[12]  George Demiris,et al.  Examining daily activity routines of older adults using workflow , 2017, J. Biomed. Informatics.

[13]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[14]  E. Protas,et al.  Relationship of falls and fear of falling to activity limitations and physical inactivity in Parkinson's disease. , 2015, Journal of aging and physical activity.

[15]  Usman Naeem,et al.  Recognition of Activities of Daily Living from Topic Model , 2016, EUSPN/ICTH.

[16]  Joel J. P. C. Rodrigues,et al.  Mobile-health: A review of current state in 2015 , 2015, J. Biomed. Informatics.

[17]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[18]  Laura L. Pullum,et al.  Appendix A: References. , 2022, The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease.

[19]  Kimiaki Shirahama,et al.  Comparison of Feature Learning Methods for Human Activity Recognition Using Wearable Sensors , 2018, Sensors.

[20]  Julia Adler-Milstein,et al.  Reducing Medicare Spending Through Electronic Health Information Exchange: The Role of Incentives and Exchange Maturity , 2018, Inf. Syst. Res..

[21]  M. Haghi,et al.  Wearable Devices in Medical Internet of Things: Scientific Research and Commercially Available Devices , 2017, Healthcare informatics research.

[22]  Debi Prosad Dogra,et al.  deepGesture: Deep learning-based gesture recognition scheme using motion sensors , 2018, Displays.

[23]  Hsinchun Chen,et al.  Healthcare Predictive Analytics for Risk Profiling in Chronic Care: A Bayesian Multitask Learning Approach , 2017, MIS Q..

[24]  Lin Tan,et al.  Sequential Human Activity Recognition Based on Deep Convolutional Network and Extreme Learning Machine Using Wearable Sensors , 2018, J. Sensors.

[25]  Magda Tsolaki,et al.  Mild cognitive impairment and deficits in instrumental activities of daily living: a systematic review , 2015, Alzheimer's Research & Therapy.

[26]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[27]  Nuno M. Garcia,et al.  Identification of activities of daily living through data fusion on motion and magnetic sensors embedded on mobile devices , 2018, Pervasive Mob. Comput..

[28]  Enamul Hoque,et al.  Home wireless sensing system for monitoring nighttime agitation and incontinence in patients with Alzheimer's disease , 2015, Wireless Health.

[29]  Diane J. Cook,et al.  Ambient intelligence for health environments , 2016, J. Biomed. Informatics.

[30]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[31]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[32]  Fitriani,et al.  Generic motion gesture detection scheme using only a triaxial accelerometer , 2011, 2011 IEEE 15th International Symposium on Consumer Electronics (ISCE).

[33]  Hock-Hai Teo,et al.  Comparing Potential and Actual Innovators: An Empirical Study of Mobile Data Services Innovation , 2015, MIS Q..

[34]  Travis E. Oliphant,et al.  Guide to NumPy , 2015 .

[35]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[36]  Elizabeth J. Davidson,et al.  Unity in Diversity: Electronic Patient Record Use in Multidisciplinary Practice , 2011, Inf. Syst. Res..

[37]  V. Conn,et al.  Interventions to Improve Medication Adherence in Hypertensive Patients: Systematic Review and Meta-analysis , 2015, Current Hypertension Reports.

[38]  Ujjwal Bhattacharya,et al.  Online Handwriting Recognition Using Levenshtein Distance Metric , 2013, 2013 12th International Conference on Document Analysis and Recognition.

[39]  Steven J. Pentland,et al.  A Video-Based Screening System for Automated Risk Assessment Using Nuanced Facial Features , 2017, J. Manag. Inf. Syst..

[40]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[41]  Ken Kelley,et al.  When Do IT Security Investments Matter? Accounting for the Influence of Institutional Factors in the Context of Healthcare Data Breaches , 2017, MIS Q..

[42]  Wonseok Oh,et al.  Excessive Dependence on Mobile Social Apps: A Rational Addiction Perspective , 2016, Inf. Syst. Res..

[43]  Jae-Young Pyun,et al.  Deep Recurrent Neural Networks for Human Activity Recognition , 2017, Sensors.

[44]  Xiaoli Li,et al.  An integrated framework for human activity classification , 2012, UbiComp.

[45]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[46]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[47]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[48]  Hsinchun Chen,et al.  Hidden Markov Model-Based Fall Detection With Motion Sensor Orientation Calibration: A Case for Real-Life Home Monitoring , 2018, IEEE Journal of Biomedical and Health Informatics.

[49]  Marion F Walker,et al.  Interventions to reduce dependency in personal activities of daily living in community dwelling adults who use homecare services: a systematic review , 2015, Clinical rehabilitation.

[50]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[51]  Diane Kelly,et al.  Methods for Evaluating Interactive Information Retrieval Systems with Users , 2009, Found. Trends Inf. Retr..

[52]  Life expectancy increased by 5 years since 2000, but health inequalities persist , 2016, Saudi Medical Journal.

[53]  Jay F. Nunamaker,et al.  Systems Development in Information Systems Research , 1990, J. Manag. Inf. Syst..

[54]  Ahmad Almogren,et al.  A robust human activity recognition system using smartphone sensors and deep learning , 2018, Future Gener. Comput. Syst..

[55]  Hua Ye,et al.  User Service Innovation on Mobile Phone Platforms: Investigating Impacts of Lead Userness, Toolkit Support, and Design Autonomy , 2018, MIS Q..

[56]  Juhee Kwon,et al.  Proactive Versus Reactive Security Investments in the Healthcare Sector , 2014, MIS Q..

[57]  Thomas Plötz,et al.  Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables , 2016, IJCAI.

[58]  Inderpal Singh,et al.  Assessment and management of dementia in the general hospital setting , 2014 .

[59]  Hsinchun Chen,et al.  A sequence-to-sequence model-based deep learning approach for recognizing activity of daily living for senior care , 2018, J. Biomed. Informatics.

[60]  ZhouLina,et al.  The effects of tree-view based presentation adaptation on mobile web browsing , 2011 .

[61]  Petr Sojka,et al.  Software Framework for Topic Modelling with Large Corpora , 2010 .

[62]  Subhajyoti Bandyopadhyay,et al.  An Analysis of the Adoption of Digital Health Records under Switching Costs , 2010, Inf. Syst. Res..

[63]  John A. Stankovic,et al.  SARRIMA: smart ADL recognizer and resident identifier in multi-resident accommodations , 2015, Wireless Health.

[64]  Indranil R. Bardhan,et al.  Predictive Analytics for Readmission of Patients with Congestive Heart Failure , 2015, Inf. Syst. Res..

[65]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[66]  Param Vir Singh,et al.  Learning Curves of Agents with Diverse Skills in Information Technology-Enabled Physician Referral Systems , 2011, Inf. Syst. Res..

[67]  Bandar Saleh Mouhammed ِAlmaslukh,et al.  An effective deep autoencoder approach for online smartphone-based human activity recognition , 2017 .

[68]  Yuqing Chen,et al.  A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[69]  E. Glickman,et al.  Activities of Daily Living , 2020, Encyclopedia of Autism Spectrum Disorders.

[70]  Ruth Jemma White Using topic models to detect behaviour patterns for healthcare monitoring , 2018 .

[71]  Jesse Hoey,et al.  Sensor-Based Activity Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[72]  Viswanath Venkatesh,et al.  Combating Infant Mortality in Rural India: Evidence from a Field Study of eHealth Kiosk Implementations , 2016, MIS Q..

[73]  K. Jarrod Millman,et al.  Python for Scientists and Engineers , 2011, Comput. Sci. Eng..

[74]  Bernt Schiele,et al.  Discovery of activity patterns using topic models , 2008 .

[75]  Qingzhong Liu,et al.  Smartphone Sensor-Based Activity Recognition by Using Machine Learning and Deep Learning Algorithms , 2018 .

[76]  Hsinchun Chen,et al.  SilverLink: Smart Home Health Monitoring for Senior Care , 2015, ICSH.

[77]  Mohamad Khalil,et al.  Recognition of different daily living activities using hidden Markov model regression , 2016, 2016 3rd Middle East Conference on Biomedical Engineering (MECBME).

[78]  Davide Anguita,et al.  Transition-Aware Human Activity Recognition Using Smartphones , 2016, Neurocomputing.

[79]  Jay F. Nunamaker,et al.  Creating High-Value Real-World Impact through Systematic Programs of Research , 2017, MIS Q..

[80]  Vladimir I. Levenshtein,et al.  Binary codes capable of correcting deletions, insertions, and reversals , 1965 .

[81]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[82]  Juan Villegas-Cortez,et al.  Coarse-Fine Convolutional Deep-Learning Strategy for Human Activity Recognition , 2019, Sensors.