Nonintrusive Fine-Grained Home Care Monitoring: Characterizing Quality of In-Home Postural Changes Using Bone-Based Human Sensing

In contrast to the physical activities of able-bodied people at home, most people who require long-term specific care (e.g., bedridden patients and patients who have difficulty walking) usually show more low-intensity slow physical activities with postural changes. Although the existing devices can detect data such as heart rate and the number of steps, they have been increasing the physical burden relying on long-term wearing. The purpose of this paper is to realize a noninvasive fine-grained home care monitoring system that is sustainable for people requiring special care. In the proposed method, we present a novel technique that integrates inexpensive camera devices and bone-based human sensing technologies to characterize the quality of in-home postural changes. We realize a local process in feature data acquisition once per second, which extends from a computer browser to Raspberry Pi. Our key idea is to regard the changes of the bounding box output by standalone pose estimation models in the shape and distance as the quality of the pose conversion, body movement, and positional changes. Furthermore, we use multiple servers to realize distributed processing that uploads data to implement home monitoring as a web service. Based on the experimental results, we conveyed our findings and advice to the subject that include where the daily living habits and the irregularity of home care timings needed improvement.

[1]  Sung-Hoon Lee,et al.  Monitoring System for the Elderly Living Alone Using the RaspberryPi Sensor , 2018 .

[2]  Hoojin Lee,et al.  Implementation of Application for Smart Healthcare Exercise Management Based on Artificial Intelligence , 2020 .

[3]  Thar Baker,et al.  Remote health monitoring of elderly through wearable sensors , 2019, Multimedia Tools and Applications.

[4]  Heedong Ko,et al.  Data-Driven Smart Home System for Elderly People Based on Web Technologies , 2016, HCI.

[5]  Masahide Nakamura,et al.  Toward Affordable and Practical Home Context Recognition: -Framework and Implementation with Image-based Cognitive API- , 2019, Int. J. Networked Distributed Comput..

[6]  Yenumula B. Reddy,et al.  Image Classification Using TensorFlow , 2019 .

[7]  Karen A Schutzer,et al.  Barriers and motivations to exercise in older adults. , 2004, Preventive medicine.

[8]  Kailas Patil,et al.  A consumer-based smart home and health monitoring system , 2018, Int. J. Comput. Appl. Technol..

[9]  Radha Shankarmani,et al.  A Virtual Trial Room using Pose Estimation and Homography , 2020, 2020 4th International Conference on Intelligent Computing and Control Systems (ICICCS).

[10]  Jaya Mishra,et al.  Objects Talk - Object Detection and Pattern Tracking Using TensorFlow , 2018, 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT).

[11]  Kahlid Soofi,et al.  Image Classification , 2008, Encyclopedia of Multimedia.

[12]  Diane J. Cook,et al.  Wearable Device-Independent Next Day Activity and Next Night Sleep Prediction for Rehabilitation Populations , 2020, IEEE Journal of Translational Engineering in Health and Medicine.

[13]  Ahmad Jalal,et al.  Vision-Based Human Activity Recognition System Using Depth Silhouettes: A Smart Home System for Monitoring the Residents , 2019, Journal of Electrical Engineering & Technology.

[14]  Angela Patricia Bacelis-Rivero,et al.  Assessment of Physical Activity in Adults: A Review of Validated Questionnaires From a Nutritionist’s Point of View , 2020, Evaluation & the health professions.

[15]  Hao Jiang,et al.  Visual-GPS: Ego-Downward and Ambient Video Based Person Location Association , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[16]  Masahide Nakamura,et al.  Toward Flexible and Efficient Home Context Sensing: Capability Evaluation and Verification of Image-Based Cognitive APIs † , 2020, Sensors.

[17]  Sheroz Khan,et al.  Daily activity monitoring of an elderly Person for determining their wellness , 2016 .

[18]  Janet Wesson,et al.  A Smart Home Environment to Support Risk Monitoring for the Elderly Living Independently , 2019, South Afr. Comput. J..

[19]  Juergen Gall,et al.  PoseTrack: Joint Multi-person Pose Estimation and Tracking , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Lorenzo Torresani,et al.  Detect-and-Track: Efficient Pose Estimation in Videos , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[21]  Roque Marín,et al.  Monitoring elderly people at home with temporal Case-Based Reasoning , 2017, Knowl. Based Syst..

[22]  Mark Everingham,et al.  Clustered Pose and Nonlinear Appearance Models for Human Pose Estimation , 2010, BMVC.

[23]  Ibrahim Almarashdeh,et al.  Real-Time Elderly Healthcare Monitoring Expert System Using Wireless Sensor Network , 2018, ArXiv.

[24]  Teerakiat Kerdcharoen,et al.  Social data shoes for gait monitoring of elderly people in smart home , 2016, 2016 9th Biomedical Engineering International Conference (BMEiCON).

[25]  Rubén González Crespo,et al.  Computational method for monitoring pauses exercises in office workers through a vision model , 2020, Journal of Ambient Intelligence and Humanized Computing.

[26]  Amir Masoud Rahmani,et al.  Structural and behavioral reference model for IoT-based elderly health-care systems in smart home , 2019, Int. J. Commun. Syst..

[27]  Kholoud Maswadi,et al.  Systematic Literature Review of Smart Home Monitoring Technologies Based on IoT for the Elderly , 2020, IEEE Access.

[28]  B. B. Zaidan,et al.  Smart Home-based IoT for Real-time and Secure Remote Health Monitoring of Triage and Priority System using Body Sensors: Multi-driven Systematic Review , 2019, Journal of Medical Systems.

[29]  Yaser Sheikh,et al.  Efficient Online Multi-Person 2D Pose Tracking With Recurrent Spatio-Temporal Affinity Fields , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Zimin Xu,et al.  Sensor-based activity recognition of solitary elderly via stigmergy and two-layer framework , 2020, Eng. Appl. Artif. Intell..

[31]  Tao Song,et al.  A deep model method for recognizing activities of workers on offshore drilling platform by multistage convolutional pose machine , 2020 .

[32]  M. Ikenaga,et al.  Association of Lower-Extremity Muscle Performance and Physical Activity Level and Intensity in Middle-Aged and Older Adults: A Doubly Labeled Water and Accelerometer Study , 2020, The journal of nutrition, health & aging.

[33]  Cees G. M. Snoek,et al.  Actor-Transformers for Group Activity Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Masahide Nakamura,et al.  Integrating Multiple Models Using Image-as-Documents Approach for Recognizing Fine-Grained Home Contexts † , 2020, Sensors.

[35]  Ning An,et al.  A Pilot Study on a Smart Home for Elders Based on Continuous In-Home Unobtrusive Monitoring Technology , 2019, HERD.

[36]  Dympna O'Sullivan,et al.  Monitoring meaningful activities using small low-cost devices in a smart home , 2019, Personal and Ubiquitous Computing.

[37]  D. Skelton,et al.  Effects of physical activity on postural stability. , 2001, Age and ageing.

[38]  Jie Yu,et al.  Beyond 2D: Fusion of Monocular 3D Pose, Motion and Appearance for Human Action Recognition , 2019, 2019 22th International Conference on Information Fusion (FUSION).

[39]  Kyoung Mu Lee,et al.  Camera Distance-Aware Top-Down Approach for 3D Multi-Person Pose Estimation From a Single RGB Image , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[40]  Kathleen S. Thomas,et al.  How Does Physical Activity Impact Postural Stability , 2014 .

[41]  Minggang Shao,et al.  A Remote Health Monitoring System for the Elderly Based on Smart Home Gateway , 2017, Journal of healthcare engineering.

[42]  Yali Amit,et al.  Object Detection , 2020, Computer Vision, A Reference Guide.

[43]  Dashrath Mane,et al.  Image File Security using Base64 Algorithm , 2014 .

[44]  J. Speakman,et al.  Comparison of total and activity energy expenditure estimates from physical activity questionnaires and doubly labelled water: a systematic review and meta-analysis , 2020, British Journal of Nutrition.

[45]  Fabian Camilo Herrera Ramírez,et al.  FabRigo: modelo informático para el monitoreo y control de rutinas de pausas activas en trabajadores de oficina por medio del modelo posenet , 2019 .

[46]  S. Hansun A new approach of moving average method in time series analysis , 2013, 2013 Conference on New Media Studies (CoNMedia).

[47]  M. Shamim Hossain,et al.  Patient status monitoring for smart home healthcare , 2016, 2016 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[48]  Marie-Odile Berger,et al.  Pose Estimation for , 2002 .

[49]  Radosveta Sokullu,et al.  IoT supported smart home for the elderly , 2020, Internet Things.

[50]  Yosuke Yamada,et al.  Total Energy Expenditure, Body Composition, Physical Activity, and Step Count in Japanese Preschool Children: A Study Based on Doubly Labeled Water , 2020, Nutrients.

[51]  Valentina Bianchi,et al.  A Plug and Play IoT Wi-Fi Smart Home System for Human Monitoring , 2018, Electronics.

[52]  Olfa Jemai,et al.  PTLHAR: PoseNet and transfer learning for human activities recognition based on body articulations , 2020, International Conference on Machine Vision.