RECOGNITION AND CLASSIFICATION OF HUMAN ACTIVITIES USING WEARABLE SENSORS

RECOGNITION AND CLASSIFICATION OF HUMAN ACTIVITIES USING WEARABLE SENSORS Aras Yurtman M.S. in Electrical and Electronics Engineering Supervisor: Prof. Dr. Billur Barshan September 2012 We address the problem of detecting and classifying human activities using two different types of wearable sensors. In the first part of the thesis, a comparative study on the different techniques of classifying human activities using tag-based radio-frequency (RF) localization is provided. Position data of multiple RF tags worn on the human body are acquired asynchronously and non-uniformly. Curves fitted to the data are re-sampled uniformly and then segmented. The effect of varying the relevant system parameters on the system accuracy is investigated. Various curve-fitting, segmentation, and classification techniques are compared and the combination resulting in the best performance is presented. The classifiers are validated through the use of two different cross-validation methods. For the complete classification problem with 11 classes, the proposed system demonstrates an average classification error of 8.67% and 21.30% for 5-fold and subject-based leave-one-out (L1O) cross validation, respectively. When the number of classes is reduced to five by omitting the transition classes, these errors become 1.12% and 6.52%. The system demonstrates acceptable classification performance despite that tag-based RF localization does not provide very accurate position measurements. In the second part, data acquired from five sensory units worn on the human body, each containing a tri-axial accelerometer, a gyroscope, and a magnetometer, during 19 different human activities are used to calculate inter-subject and interactivity variations in the data with different methods. Absolute, Euclidean, and dynamic time-warping (DTW) distances are used to assess the similarity of the signals. The comparisons are made using time-domain data and feature vectors. Different normalization methods are used and compared. The “best” subject is defined and identified according to his/her average distance to the other subjects.

[1]  Hassan Ghasemzadeh,et al.  Distributed Continuous Action Recognition Using a Hidden Markov Model in Body Sensor Networks , 2009, DCOSS.

[2]  Angelo Maria Sabatini,et al.  Inertial sensing in biomechanics: a survey of computational techniques bridging motion analysis and personal navigation , 2006 .

[3]  Mitja Lustrek,et al.  Fall Detection and Activity Recognition with Machine Learning , 2009, Informatica.

[4]  Matthias Kranz,et al.  Sensor-Based Skill Assessment for Health and Fitness Applications , 2011 .

[5]  Wolfram Burgard,et al.  Mapping and localization with RFID technology , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[6]  Hector M. Garcia,et al.  Virtual Reality in Gait Rehabilitation , 2007 .

[7]  Matjaz Gams,et al.  Behavior Analysis Based on Coordinates of Body Tags , 2009, AmI.

[8]  Billur Barshan,et al.  Leg Motion Classification with Artificial Neural Networks Using Wavelet-Based Features of Gyroscope Signals , 2011, Sensors.

[9]  David Wetherall,et al.  Recognizing daily activities with RFID-based sensors , 2009, UbiComp.

[10]  B. Ugur Toreyin,et al.  Falling Person Detection Using Multi-sensor Signal Processing , 2007 .

[11]  Wai Yin Wong,et al.  Clinical Applications of Sensors for Human Posture and Movement Analysis: A Review , 2007, Prosthetics and orthotics international.

[12]  Kamiar Aminian,et al.  Mobility assessment in older people: new possibilities and challenges , 2007, European journal of ageing.

[13]  Patrick Boissy,et al.  Accelerometer-based wireless body area network to estimate intensity of therapy in post-acute rehabilitation , 2008, Journal of NeuroEngineering and Rehabilitation.

[14]  Paul Lukowicz,et al.  Gesture spotting with body-worn inertial sensors to detect user activities , 2008, Pattern Recognit..

[15]  Toni Giorgino,et al.  Wireless Support to Poststroke Rehabilitation: MyHeart's Neurological Rehabilitation Concept , 2009, IEEE Transactions on Information Technology in Biomedicine.

[16]  Yunhao Liu,et al.  LANDMARC: Indoor Location Sensing Using Active RFID , 2004, Proceedings of the First IEEE International Conference on Pervasive Computing and Communications, 2003. (PerCom 2003)..

[17]  Merryn J Mathie,et al.  Accelerometry: providing an integrated, practical method for long-term, ambulatory monitoring of human movement , 2004, Physiological measurement.

[18]  Huosheng Hu,et al.  Integration of Vision and Inertial Sensors for Home-based Rehabilitation , 2004 .

[19]  Gita Reese Sukthankar,et al.  A cost minimization approach to human behavior recognition , 2005, AAMAS '05.

[20]  R. Willmann,et al.  Home Stroke Rehabilitation for the Upper Limbs , 2007, 2007 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[21]  Matjaz Gams,et al.  An Agent-Based Approach to Care in Independent Living , 2010, AmI.

[22]  Toni Giorgino,et al.  Matching incomplete time series with dynamic time warping: an algorithm and an application to post-stroke rehabilitation , 2009, Artif. Intell. Medicine.

[23]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[24]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[25]  Sandra Dominikus,et al.  Strong Authentication for RFID Systems Using the AES Algorithm , 2004, CHES.

[26]  Huiru Zheng,et al.  Developing a telemonitoring system for stroke rehabilitation , 2007 .

[27]  Nizam Uddin Ahamed,et al.  Biosensors assisted automated rehabilitation systems: A systematic review , 2012 .

[28]  Jake K. Aggarwal,et al.  Human Motion Analysis: A Review , 1999, Comput. Vis. Image Underst..

[29]  P. Boissy,et al.  In-home telerehabilitation for geriatric patients , 2008, IEEE Engineering in Medicine and Biology Magazine.

[30]  Ashok Veeraraghavan,et al.  The Function Space of an Activity , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[31]  Jing Liu,et al.  Survey of Wireless Indoor Positioning Techniques and Systems , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[32]  Yang Yang,et al.  Automated Recognition of Sequential Patterns in Captured Motion Streams , 2010, WAIM.

[33]  Albrecht Schmidt,et al.  Multi-sensor Activity Context Detection for Wearable Computing , 2003, EUSAI.

[34]  Shahnawaz Anwer,et al.  Effectiveness of electromyographic biofeedback training on quadriceps muscle strength in osteoarthritis of knee , 2011 .

[35]  Billur Barshan,et al.  Inter- and intra-subject variations in activity recognition using inertial sensors and magnetometers , 2012 .

[36]  José Bravo,et al.  m-Physio: Personalized Accelerometer-based Physical Rehabilitation Platform , 2010 .

[37]  Huosheng Hu,et al.  Inertial motion tracking of human arm movements in stroke rehabilitation , 2005, IEEE International Conference Mechatronics and Automation, 2005.

[38]  Billur Barshan,et al.  Comparative study on classifying human activities with miniature inertial and magnetic sensors , 2010, Pattern Recognit..

[39]  Joshua R. Smith,et al.  RFID-based techniques for human-activity detection , 2005, Commun. ACM.

[40]  Aleksandar Milenkovic,et al.  Journal of Neuroengineering and Rehabilitation Open Access a Wireless Body Area Network of Intelligent Motion Sensors for Computer Assisted Physical Rehabilitation , 2005 .

[41]  Billur Barshan,et al.  Pedestrian dead reckoning employing simultaneous activity recognition cues , 2012 .

[42]  Ju-Jang Lee,et al.  Localization and map-building of mobile robot based on RFID sensor fusion system , 2008, 2008 6th IEEE International Conference on Industrial Informatics.

[43]  Svetha Venkatesh,et al.  Activity recognition and abnormality detection with the switching hidden semi-Markov model , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[44]  M. Decramer,et al.  Quantifying physical activity in daily life with questionnaires and motion sensors in COPD , 2006, European Respiratory Journal.

[45]  Billur Barshan,et al.  Investigation of personal variations in activity recognition using miniature inertial sensors and magnetometers , 2012, 2012 20th Signal Processing and Communications Applications Conference (SIU).

[46]  Billur Barshan,et al.  Human activity recognition using tag-based localization , 2012, 2012 20th Signal Processing and Communications Applications Conference (SIU).

[47]  Kuan-Rong Lee,et al.  A flexible sequence alignment approach on pattern mining and matching for human activity recognition , 2010, Expert Syst. Appl..

[48]  Billur Barshan,et al.  Classifying Human Leg Motions with Uniaxial Piezoelectric Gyroscopes , 2009, Sensors.

[49]  Pete Steggles,et al.  THE UBISENSE SMART SPACE PLATFORM , 2005 .

[50]  Jan Brutovsky,et al.  Low-cost motivated rehabilitation system for post-operation exercises , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[51]  Huosheng Hu,et al.  A Novel Sensing and Data Fusion System for 3-D Arm Motion Tracking in Telerehabilitation , 2008, IEEE Transactions on Instrumentation and Measurement.

[52]  Eric Garshick,et al.  Free-living physical activity in COPD: assessment with accelerometer and activity checklist. , 2009, Journal of rehabilitation research and development.

[53]  Emil Jovanov,et al.  An accelerometer-based physical rehabilitation system , 2002, Proceedings of the Thirty-Fourth Southeastern Symposium on System Theory (Cat. No.02EX540).

[54]  Stephen A. Weis RFID (Radio-Frequency Identification) , 2012 .

[55]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[56]  Jian Lu,et al.  Sensor-Based Human Activity Recognition in a Multi-user Scenario , 2009, AmI.

[57]  Adrian Hilton,et al.  A survey of advances in vision-based human motion capture and analysis , 2006, Comput. Vis. Image Underst..

[58]  Jennifer Healey,et al.  A Long-Term Evaluation of Sensing Modalities for Activity Recognition , 2007, UbiComp.

[59]  Bernt Schiele,et al.  ADL recognition based on the combination of RFID and accelerometer sensing , 2008, 2008 Second International Conference on Pervasive Computing Technologies for Healthcare.

[60]  Eamonn J. Keogh,et al.  Accelerating Dynamic Time Warping Subsequence Search with GPUs and FPGAs , 2010, 2010 IEEE International Conference on Data Mining.

[61]  Greg Welch,et al.  Motion Tracking: No Silver Bullet, but a Respectable Arsenal , 2002, IEEE Computer Graphics and Applications.

[62]  Franco Zambonelli,et al.  Detecting activities from body-worn accelerometers via instance-based algorithms , 2010, Pervasive Mob. Comput..

[63]  James M. Rehg,et al.  A Scalable Approach to Activity Recognition based on Object Use , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[64]  Moeness G. Amin,et al.  Localization and Tracking of Passive RFID Tags Based on Direction Estimation , 2007 .

[65]  Meinard Müller,et al.  Information retrieval for music and motion , 2007 .

[66]  Roy Want,et al.  An introduction to RFID technology , 2006, IEEE Pervasive Computing.

[67]  Billur Barshan,et al.  Human Activity Recognition Using Inertial/Magnetic Sensor Units , 2010, HBU.

[68]  Toshiyo Tamura,et al.  Monitoring rehabilitation training for hemiplegic patients by using a tri-axial accelerometer , 2001, 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[69]  Angelo M. Sabatini,et al.  Machine Learning Methods for Classifying Human Physical Activity from On-Body Accelerometers , 2010, Sensors.

[70]  Tieniu Tan,et al.  Recent developments in human motion analysis , 2003, Pattern Recognit..

[71]  Yaser Sheikh,et al.  Exploring the space of a human action , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[72]  Context-Aware Computing,et al.  Inferring Activities from Interactions with Objects , 2004 .

[73]  C. Winstein,et al.  Validity of accelerometry for monitoring real-world arm activity in patients with subacute stroke: evidence from the extremity constraint-induced therapy evaluation trial. , 2006, Archives of physical medicine and rehabilitation.

[74]  Paul Fergus,et al.  Remote physiotherapy treatments using wireless body sensor networks , 2009, IWCMC.

[75]  Takeo Kanade,et al.  Classifying human motion quality for knee osteoarthritis using accelerometers , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[76]  J. Winters,et al.  Wearable sensors and telerehabilitation , 2003, IEEE Engineering in Medicine and Biology Magazine.

[77]  Huiru Zheng,et al.  Web-based monitoring system for home-based rehabilitation with stroke patients , 2005, 18th IEEE Symposium on Computer-Based Medical Systems (CBMS'05).

[78]  Paul Fergus,et al.  Body Area Networks for Movement Analysis in Physiotherapy Treatments , 2010, 2010 IEEE 24th International Conference on Advanced Information Networking and Applications Workshops.

[79]  B. Steele,et al.  Bodies in motion: monitoring daily activity and exercise with motion sensors in people with chronic pulmonary disease. , 2003, Journal of rehabilitation research and development.

[80]  M. Bouet,et al.  RFID tags: Positioning principles and localization techniques , 2008, 2008 1st IFIP Wireless Days.