MobiGesture: Mobility-aware hand gesture recognition for healthcare

Abstract Accurate recognition of hand gestures while moving is still a significant challenge, which prevents the wide use of existing gesture recognition technology. In this paper, we propose a novel mobility-aware hand gesture segmentation algorithm to detect and segment hand gestures. We also propose a Convolutional Neural Network (CNN) to classify hand gestures with mobility noises. Based on the segmentation and classification algorithms, we develop MobiGesture, a mobility-aware hand gesture recognition system for healthcare. For the leave-one-subject-out cross-validation test, experiments with human subjects show that the proposed segmentation algorithm achieves 94.0% precision, and 91.2% recall when the user is moving. The proposed hand gesture classification algorithm is 16.1%, 15.3%, and 14.4% more accurate than state-of-the-art work when the user is standing, walking and jogging, respectively.

[1]  Kyoung-Ho Kang,et al.  Self-contained spatial input device for wearable computers , 2003, Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings..

[2]  Inseok Hwang,et al.  E-Gesture: a collaborative architecture for energy-efficient gesture recognition with hand-worn sensor and mobile devices , 2011, SenSys.

[3]  Joseph A. Paradiso,et al.  An Inertial Measurement Framework for Gesture Recognition and Applications , 2001, Gesture Workshop.

[4]  Luc Noreau,et al.  Older Adults, Chronic Disease and Leisure-Time Physical Activity , 2008, Gerontology.

[5]  I-Min Lee,et al.  The importance of walking to public health. , 2008, Medicine and science in sports and exercise.

[6]  Peter Alfeld,et al.  A trivariate clough-tocher scheme for tetrahedral data , 1984, Comput. Aided Geom. Des..

[7]  W. Satariano,et al.  Physical activity, public health, and aging: critical issues and research priorities. , 2006, The journals of gerontology. Series B, Psychological sciences and social sciences.

[8]  Wen-Huang Cheng,et al.  FingerPad: private and subtle interaction using fingertips , 2013, UIST.

[9]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[10]  Evangelos Kalogerakis,et al.  RisQ: recognizing smoking gestures with inertial sensors on a wristband , 2014, MobiSys.

[11]  Kazuya Murao,et al.  A recognition method for combined activities with accelerometers , 2014, UbiComp Adjunct.

[12]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[13]  Adam W. Hoover,et al.  A New Method for Measuring Meal Intake in Humans via Automated Wrist Motion Tracking , 2012, Applied Psychophysiology and Biofeedback.

[14]  Van Nostrand,et al.  Error Bounds for Convolutional Codes and an Asymptotically Optimum Decoding Algorithm , 1967 .

[15]  Carl de Boor,et al.  A Practical Guide to Splines , 1978, Applied Mathematical Sciences.

[16]  Parth H. Pathak,et al.  Finger-writing with Smartwatch: A Case for Finger and Hand Gesture Recognition using Smartwatch , 2015, HotMobile.

[17]  Lorenzo Porzi,et al.  A smart watch-based gesture recognition system for assisting people with visual impairments , 2013, IMMPD '13.

[18]  Shuangquan Wang,et al.  Ultigesture: A wristband-based platform for continuous gesture control in healthcare , 2019, Smart Health.

[19]  Sean White,et al.  uTrack: 3D input using two magnetic sensors , 2013, UIST.

[20]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..

[21]  Shwetak N. Patel,et al.  Finexus: Tracking Precise Motions of Multiple Fingertips Using Magnetic Sensing , 2016, CHI.

[22]  Susan E. Hardy,et al.  Ability to Walk 1/4 Mile Predicts Subsequent Disability, Mortality, and Health Care Costs , 2010, Journal of General Internal Medicine.

[23]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[24]  Jennifer Balfour,et al.  Just Get Out the Door! Importance of Walking Outside the Home for Maintaining Mobility: Findings from the Women's Health and Aging Study , 2005, Journal of the American Geriatrics Society.