Linguistic Summarization of In-Home Sensor Data.

INTRODUCTION With the increase in the population of older adults around the world, a significant amount of work has been done on in-home sensor technology to aid the elderly age independently. However, due to the large amounts of data generated by the sensors, it takes a lot of effort and time for the clinicians to makes sense of this data. In this work, we develop a system to help make this data more useful by presenting it in the form of natural language. METHODS We start by identifying important attributes in the sensor data that are relevant to the health of the elderly. We then develop algorithms to extract these important health related features from the sensor parameters and summarize them in natural language. We focus on making the natural language summaries to be informative, accurate and concise. RESULTS We designed multiple surveys using real and synthetic data to validate the summaries produced by our algorithms. We show that the algorithms produce meaningful results comparable to human subjects. We also implemented our linguistic summarization system to produce summaries of data leading to health alerts derived from the sensor data. The system is running live in 110 apartments currently. By the means of retrospective case studies, we illustrate that the linguistic summaries are able to make the connection between changes in the sensor data and the health of the elderly. CONCLUSIONS We present a system that extracts important clinically relevant features from in-home sensor data generated in the apartments of the elderly and summarize those features in natural language. The preliminary testing of our summarization system shows that it has the potential to help the clinicians utilize this data effectively.

[1]  Steven J. Miller,et al.  Automated technology to speed recognition of signs of illness in older adults. , 2012, Journal of gerontological nursing.

[2]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[3]  Dimitra Gkatzia,et al.  Natural Language Generation enhances human decision-making with uncertain information , 2016, ACL.

[4]  Kevin A Hallgren,et al.  Computing Inter-Rater Reliability for Observational Data: An Overview and Tutorial. , 2012, Tutorials in quantitative methods for psychology.

[5]  G L Alexander,et al.  Generating sensor data summaries to communicate change in elders' health status. , 2014, Applied clinical informatics.

[6]  Marjorie Skubic,et al.  Developing a comprehensive electronic health record to enhance nursing care coordination, use of technology, and research. , 2010, Journal of gerontological nursing.

[7]  Eleni Stroulia,et al.  International Journal of Medical Informatics , 2016 .

[8]  Marjorie Skubic,et al.  Using Embedded Sensors in Independent Living to Predict Gait Changes and Falls , 2017, Western journal of nursing research.

[9]  Maureen Schmitter-Edgecombe,et al.  Iterative Design of Visual Analytics for a Clinician-in-the-Loop Smart Home , 2019, IEEE Journal of Biomedical and Health Informatics.

[10]  Michael Marschollek,et al.  Decision support at home (DS@HOME) – system architectures and requirements , 2012, BMC Medical Informatics and Decision Making.

[11]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[12]  Qing Zhang,et al.  Understanding Smart Home Sensor Data for Ageing in Place Through Everyday Household Routines: A Mixed Method Case Study , 2017, JMIR mHealth and uHealth.

[13]  Marjorie Skubic,et al.  Randomized Trial of Intelligent Sensor System for Early Illness Alerts in Senior Housing. , 2017, Journal of the American Medical Directors Association.

[14]  M. Yefimova Using in-Home Monitoring Technology to Identify Deviations in Daily Routines Preceding Changes in Health Trajectory of Older Adults , 2016 .

[15]  Shuang Wang,et al.  Activity Density Map Visualization and Dissimilarity Comparison for Eldercare Monitoring , 2012, IEEE Transactions on Information Technology in Biomedicine.

[16]  Dimitra Gkatzia,et al.  Data-to-Text Generation Improves Decision-Making Under Uncertainty , 2017, IEEE Computational Intelligence Magazine.

[17]  Marjorie Skubic,et al.  Evolution of an Early Illness Warning System to Monitor Frail Elders in Independent Living. , 2011, Journal of healthcare engineering.

[18]  Diane J. Cook,et al.  Modeling patterns of activities using activity curves , 2016, Pervasive Mob. Comput..

[19]  Yuval Shahar,et al.  An automated knowledge-based textual summarization system for longitudinal, multivariate clinical data , 2016, J. Biomed. Informatics.

[20]  M. Skubic,et al.  Management of Dementia and Depression Utilizing In- Home Passive Sensor Data. , 2013, Gerontechnology : international journal on the fundamental aspects of technology to serve the ageing society.

[21]  Steven J. Miller,et al.  Evaluation of Health Alerts From an Early Illness Warning System in Independent Living , 2013, Computers, informatics, nursing : CIN.

[22]  Albert Gatt,et al.  From data to text in the Neonatal Intensive Care Unit: Using NLG technology for decision support and information management , 2009, AI Commun..

[23]  George Hripcsak,et al.  Reference Standards, Judges, and Comparison Subjects , 2002 .

[24]  George Hripcsak,et al.  Review Paper: Reference Standards, Judges, and Comparison Subjects: Roles for Experts in Evaluating System Performance , 2002, J. Am. Medical Informatics Assoc..

[25]  Emiel Krahmer,et al.  Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation , 2017, J. Artif. Intell. Res..

[26]  James M. Keller,et al.  Fundamentals of Computational Intelligence: Neural Networks, Fuzzy Systems, and Evolutionary Computation , 2016 .

[27]  Susan N. Friel,et al.  Making sense of graphs: Critical factors influencing comprehension and instructional implications. , 2001 .