Recognizing Running Movement Changes with Quaternions on a Sports Watch

Sports watches are popular amongst runners but limited in terms of sensor locations (i.e., one location at the wrist). Thus, apps on the watch cannot directly sense movement changes in arbitrary body locations. This, in turn, severely limits training support services, contextual awareness and seamless interaction mechanisms. Our approach addresses this gap and connects the watch with two strategically placed inertial measurement units (IMUs) via bluetooth low energy (BLE). Our prototypical app for Wear OS receives orientation (quaternion) data and matches the sensors to the arm or leg segments using a flexible and simple procedure. We collected data from eight runners and used support vector machines (SVMs) to recognize different movements. Findings from our evaluation with different parameters indicate feasible recognition and low false-positive rates for two to three different movements, for both placements. Our approach can thus help to improve applications that support training and thus contributes to developing motion capture for personal use; it also enables movement-based interaction while running.

[1]  Xuesu Xiao,et al.  A Wearable System for Articulated Human Pose Tracking Under Uncertainty of Sensor Placement , 2018, 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob).

[2]  Jun Yang,et al.  Toward physical activity diary: motion recognition using simple acceleration features with mobile phones , 2009, IMCE '09.

[3]  Morten Fjeld,et al.  Movespace: on-body athletic interaction for running and cycling , 2018, AVI.

[4]  S. Dugan,et al.  Biomechanics and analysis of running gait. , 2005, Physical medicine and rehabilitation clinics of North America.

[5]  Wannes Meert,et al.  Fatigue Prediction in Outdoor Runners Via Machine Learning and Sensor Fusion , 2018, KDD.

[6]  Gregory D. Abowd,et al.  WatchOut: extending interactions on a smartwatch with inertial sensing , 2016, SEMWEB.

[7]  Martin J.-D. Otis,et al.  Use of a 3DOF Accelerometer for Foot Tracking and Gesture Recognition in Mobile HCI , 2013, ANT/SEIT.

[8]  Suranga Nanayakkara,et al.  Thumb-In-Motion: Evaluating Thumb-to-Ring Microgestures for Athletic Activity , 2018, SUI.

[9]  Jorunn L Helbostad,et al.  Estimation of gait cycle characteristics by trunk accelerometry. , 2004, Journal of biomechanics.

[10]  Domenico Formica,et al.  Performance Evaluation of Bluetooth Low Energy: A Systematic Review , 2017, Sensors.

[11]  Jan O. Borchers,et al.  Run&Tap: Investigation of On-Body Tapping for Runners , 2017, ISS.

[12]  Xuesu Xiao,et al.  Machine Learning for Placement-Insensitive Inertial Motion Capture , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Thomas Seel,et al.  IMU-Based Joint Angle Measurement for Gait Analysis , 2014, Sensors.

[14]  Greg Welch,et al.  Motion Tracking: No Silver Bullet, but a Respectable Arsenal , 2002, IEEE Computer Graphics and Applications.

[15]  Jasper Reenalda,et al.  Continuous three dimensional analysis of running mechanics during a marathon by means of inertial magnetic measurement units to objectify changes in running mechanics. , 2016, Journal of biomechanics.

[16]  Peter H. Veltink,et al.  Estimation of Full-Body Poses Using Only Five Inertial Sensors: An Eager or Lazy Learning Approach? , 2016, Sensors.

[17]  Salvatore Sessa,et al.  Towards Miniaturization of a MEMS-Based Wearable Motion Capture System , 2011, IEEE Transactions on Industrial Electronics.

[18]  Stefan Schneegaß,et al.  GestureSleeve: using touch sensitive fabrics for gestural input on the forearm for controlling smartwatches , 2016, SEMWEB.

[19]  Bas Van Hooren,et al.  Real-time feedback by wearables in running: Current approaches, challenges and suggestions for improvements , 2019, Journal of sports sciences.

[20]  Vivian Genaro Motti,et al.  Design Guidelines and Evaluation , 2020 .

[21]  Anthony D. Whitehead,et al.  Wearable sensor networks for motion capture , 2015, 2015 7th International Conference on Intelligent Technologies for Interactive Entertainment (INTETAIN).

[22]  Nuria Oliver,et al.  MPTrain: a mobile, music and physiology-based personal trainer , 2006, Mobile HCI.

[23]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[24]  Nancy W Glynn,et al.  Stride variability measures derived from wrist- and hip-worn accelerometers. , 2017, Gait & posture.

[25]  Y. Oshman,et al.  Averaging Quaternions , 2007 .

[26]  Philipp Scholl,et al.  Real-Time and Embedded Detection of Hand Gestures with an IMU-Based Glove , 2018, Informatics.

[27]  M. Urban,et al.  Recognition of arm gestures using multiple orientation sensors: repeatability assessment , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[28]  Agata Brajdic,et al.  Walk detection and step counting on unconstrained smartphones , 2013, UbiComp.

[29]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[30]  Deying Li,et al.  Context Impacts in Accelerometer-Based Walk Detection and Step Counting , 2018, Sensors.

[31]  Rodrigo de Oliveira,et al.  TripleBeat: enhancing exercise performance with persuasion , 2008, Mobile HCI.

[32]  Vassilis Kostakos,et al.  Running gestures: hands-free interaction during physical activity , 2010, UbiComp '10 Adjunct.

[33]  Yuanchun Shi,et al.  Designing and Evaluating Hand-to-Hand Gestures with Dual Commodity Wrist-Worn Devices , 2020, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[34]  Jun-Ming Lu,et al.  Real-Time Gait Cycle Parameter Recognition Using a Wearable Accelerometry System , 2011, Sensors.

[35]  Jian Song,et al.  An Energy Efficient Smartphone Pedometer Based on an Auto-Correlation Analysis , 2018, 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications (ISPA/IUCC/BDCloud/SocialCom/SustainCom).

[36]  Guoliang Xing,et al.  RunBuddy: a smartphone system for running rhythm monitoring , 2015, UbiComp.

[37]  Maria João Nicolau,et al.  Autocorrelation analysis of accelerometer signal to detect and count steps of smartphone users , 2019, 2019 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[38]  Joseph J. LaViola,et al.  3D Gestural Interaction: The State of the Field , 2013 .

[39]  Lucy E. Dunne,et al.  An analysis of the variability of anatomical body references within ready-to-wear garment sizes , 2016, SEMWEB.

[40]  Gernot Bauer,et al.  Live-feedback from the IMUs: animated 3D visualization for everyday-exercising , 2016, UbiComp Adjunct.

[41]  Jeffrey T. Hansberger,et al.  Real-Time Gesture Recognition Using 3D Sensory Data and a Light Convolutional Neural Network , 2019, ACM Multimedia.

[42]  Noel E. O'Connor,et al.  Kinect vs. Low-cost Inertial Sensing for Gesture Recognition , 2014, MMM.

[43]  Florian Hönig,et al.  Smart Annotation of Cyclic Data Using Hierarchical Hidden Markov Models , 2017, Sensors.

[44]  Jianke Zhu,et al.  AI Coach: Deep Human Pose Estimation and Analysis for Personalized Athletic Training Assistance , 2019, ACM Multimedia.

[45]  Miguel A. Labrador,et al.  A Smartphone-Based System for Clinical Gait Assessment , 2016, 2016 IEEE International Conference on Smart Computing (SMARTCOMP).

[46]  Gerhard Tröster,et al.  Monitoring Kinematic Changes With Fatigue in Running Using Body-Worn Sensors , 2012, IEEE Transactions on Information Technology in Biomedicine.

[47]  Baoqi Huang,et al.  Accurately Counting Steps of the Pedestrian with Varying Walking Speeds , 2018, 2018 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[48]  J. Collomosse,et al.  Real-Time Full-Body Motion Capture from Video and IMUs , 2017, 2017 International Conference on 3D Vision (3DV).

[49]  Bharti Bansal,et al.  Gesture Recognition: A Survey , 2016 .

[50]  Novacheck,et al.  The biomechanics of running. , 1998, Gait & posture.

[51]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[52]  Christian Kray,et al.  Running with drones: desired services and control gestures , 2018, OZCHI.

[53]  Michael D. Jones,et al.  Understanding How Non-Experts Collect and Annotate Activity Data , 2018, UbiComp/ISWC Adjunct.

[54]  Anthony D. Whitehead,et al.  Quaternion-Based Gesture Recognition Using Wireless Wearable Motion Capture Sensors , 2016, Sensors.

[55]  Miguel A. Labrador,et al.  Use of Wearable Sensor Technology in Gait, Balance, and Range of Motion Analysis , 2019, Applied Sciences.

[56]  B. Higginson,et al.  Methods of Running Gait Analysis , 2009, Current sports medicine reports.