Sit-to-Stand Analysis in the Wild Using Silhouettes for Longitudinal Health Monitoring

We present the first fully automated Sit-to-Stand or Stand-to-Sit (StS) analysis framework for long-term monitoring of patients in free-living environments using video silhouettes. Our method adopts a coarse-to-fine time localisation approach, where a deep learning classifier identifies possible StS sequences from silhouettes, and a smart peak detection stage provides fine localisation based on 3D bounding boxes. We tested our method on data from real homes of participants and monitored patients undergoing total hip or knee replacement. Our results show 94.4% overall accuracy in the coarse localisation and an error of 0.026 m/s in the speed of ascent measurement, highlighting important trends in the recuperation of patients who underwent surgery.

[1]  P. Olivier,et al.  Accuracy of the Microsoft Kinect sensor for measuring movement in people with Parkinson's disease. , 2014, Gait & posture.

[2]  Peter A. Flach,et al.  Smart homes, private homes? An empirical study of technology researchers’ perceptions of ethical issues in developing smart-home health technologies , 2017, BMC medical ethics.

[3]  Wolfgang L. Zagler,et al.  Ambient Assisted Living Systems - The Conflicts between Technology, Acceptance, Ethics and Privacy , 2007, Assisted Living Systems - Models, Architectures and Engineering Approaches.

[4]  B. Guthrie,et al.  Process evaluation of the data-driven quality improvement in primary care (DQIP) trial: active and less active ingredients of a multi-component complex intervention to reduce high-risk primary care prescribing , 2017, Implementation Science.

[5]  Athanase Benetos,et al.  FIVE TIMES SIT TO STAND TEST IS A PREDICTOR OF RECURRENT FALLS IN HEALTHY COMMUNITY‐LIVING SUBJECTS AGED 65 AND OLDER , 2008, Journal of the American Geriatrics Society.

[6]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[7]  Martina Ziefle,et al.  Medical Technology in Smart Homes: Exploring the User's Perspective on Privacy, Intimacy and Trust , 2011, 2011 IEEE 35th Annual Computer Software and Applications Conference Workshops.

[8]  Russell Baker,et al.  A Data Set of Human Body Movements for Physical Rehabilitation Exercises , 2018, Data.

[9]  Andrew Kerr,et al.  Frequency of the sit to stand task: An observational study of free-living adults. , 2010, Applied ergonomics.

[10]  Niall Twomey,et al.  Bridging e-Health and the Internet of Things: The SPHERE Project , 2015, IEEE Intelligent Systems.

[11]  Nathalie Japkowicz,et al.  The class imbalance problem: A systematic study , 2002, Intell. Data Anal..

[12]  Veralia Gabriela Sánchez,et al.  ETHICS OF SMART HOUSE WELFARE TECHNOLOGY FOR OLDER ADULTS: A SYSTEMATIC LITERATURE REVIEW , 2017, International Journal of Technology Assessment in Health Care.

[13]  Michael R Whitehouse,et al.  Using home sensing technology to assess outcome and recovery after hip and knee replacement in the UK: the HEmiSPHERE study protocol , 2018, BMJ Open.

[14]  Po-Yin Chen,et al.  Can sit-to-stand lower limb muscle power predict fall status? , 2014, Gait & posture.

[15]  R W Bohannon,et al.  Sit-to-Stand Test for Measuring Performance of Lower Extremity Muscles , 1995, Perceptual and motor skills.

[16]  S. Demura,et al.  Center of gravity transfer velocity during sit-to-stand is closely related to physical functions regarding fall experience of the elderly living in community dwelling , 2013 .

[17]  Philip K Schot,et al.  Sit-to-Stand Performance of Older Adults following Strength Training , 2003, Research quarterly for exercise and sport.

[18]  Ruzena Bajcsy,et al.  Vision-based Event Detection of the Sit-to-Stand Transition , 2015, EAI Endorsed Trans. Ambient Syst..

[19]  Naohiko Kanemura,et al.  Sit-to-Walk versus Sit-to-Stand or Gait Initiation: Biomechanical Analysis of Young Men , 2006 .

[20]  Andrew Zisserman,et al.  Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Majid Mirmehdi,et al.  CaloriNet: From silhouettes to calorie estimation in private environments , 2018, BMVC.

[22]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  S. Lord,et al.  Kinect-Based Five-Times-Sit-to-Stand Test for Clinical and In-Home Assessment of Fall Risk in Older People , 2015, Gerontology.

[24]  Yen-Ko Lin,et al.  Development and pilot testing of an informed consent video for patients with limb trauma prior to debridement surgery using a modified Delphi technique , 2017, BMC medical ethics.