Framework for Human Activity Recognition on Smartphones and Smartwatches

As activity recognition becomes an integral part of many mobile applications, its requirement for lightweight and accurate techniques leads to development of new tools and algorithms. This paper has three main contributions: (1) to design an architecture for automatic data collection, thus reducing the time and cost and making the process of developing new activity recognition techniques convenient for software developers as well as for the end users; (2) to develop new algorithm for activity recognition based on Long Short Term Memory networks, which is able to learn features from raw accelerometer data, completely bypassing the process of generating hand-crafted features; and (3) to investigate which combinations of smartphone and smartwatch sensors gives the best results for the activity recognition problem, i.e. to analyze if the accuracy benefits of those combinations are greater than the additional costs for combining those sensors.

[1]  Kire Trivodaliev,et al.  Internet of Things Framework for Home Care Systems , 2017, Wirel. Commun. Mob. Comput..

[2]  Michael Beigl,et al.  Energy-Efficient Activity Recognition Using Prediction , 2012, 2012 16th International Symposium on Wearable Computers.

[3]  J. Manson,et al.  The joint effects of physical activity and body mass index on coronary heart disease risk in women. , 2008, Archives of internal medicine.

[4]  Paul J. M. Havinga,et al.  Activity Recognition Using Inertial Sensing for Healthcare, Wellbeing and Sports Applications: A Survey , 2010, ARCS Workshops.

[5]  Martha E. Pollack,et al.  Autominder: an intelligent cognitive orthotic system for people with memory impairment , 2003, Robotics Auton. Syst..

[6]  Xi Long,et al.  Single-accelerometer-based daily physical activity classification , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[7]  Gary M. Weiss,et al.  Activity recognition using cell phone accelerometers , 2011, SKDD.

[8]  Mile Jovanov,et al.  Real time human activity recognition on smartphones using LSTM networks , 2018, 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[9]  J. Tuomilehto,et al.  Relationship of Physical Activity and Body Mass Index to the Risk of Hypertension: A Prospective Study in Finland , 2004, Hypertension.

[10]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[11]  Lei Liu,et al.  Human Daily Activity Recognition for Healthcare Using Wearable and Visual Sensing Data , 2016, 2016 IEEE International Conference on Healthcare Informatics (ICHI).

[12]  Davide Anguita,et al.  Energy Efficient Smartphone-Based Activity Recognition Using Fixed-Point Arithmetic , 2013 .

[13]  Tara N. Sainath,et al.  Convolutional, Long Short-Term Memory, fully connected Deep Neural Networks , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Mikkel Baun Kjærgaard,et al.  Smart Devices are Different: Assessing and MitigatingMobile Sensing Heterogeneities for Activity Recognition , 2015, SenSys.

[15]  I. Vuori,et al.  Association of leisure time physical activity with the risk of coronary heart disease, hypertension and diabetes in middle-aged men and women. , 1997, International journal of epidemiology.

[16]  Jeen-Shing Wang,et al.  Using acceleration measurements for activity recognition: An effective learning algorithm for constructing neural classifiers , 2008, Pattern Recognit. Lett..

[17]  Qiang Yang,et al.  Sensor-Based Abnormal Human-Activity Detection , 2008, IEEE Transactions on Knowledge and Data Engineering.

[18]  Jürgen Schmidhuber,et al.  Learning to Forget: Continual Prediction with LSTM , 2000, Neural Computation.

[19]  Yanjun Jia,et al.  Diatetic and Exercise Therapy against Diabetes Mellitus , 2009, 2009 Second International Conference on Intelligent Networks and Intelligent Systems.

[20]  Shumin Zhai,et al.  Long short term memory neural network for keyboard gesture decoding , 2015, 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[21]  Diane J. Cook,et al.  Human Activity Recognition and Pattern Discovery , 2010, IEEE Pervasive Computing.

[22]  Davide Anguita,et al.  Constraint-Aware Data Analysis on Mobile Devices: An Application to Human Activity Recognition on Smartphones , 2017 .

[23]  Jae-Young Pyun,et al.  Deep Recurrent Neural Networks for Human Activity Recognition , 2017, Sensors.

[24]  Eftim Zdravevski,et al.  Automatic machine-learning based identification of jogging periods from accelerometer measurements of adolescents under field conditions , 2017, PloS one.

[25]  Biljana Risteska Stojkoska,et al.  Data compression for energy efficient IoT solutions , 2017, 2017 25th Telecommunication Forum (TELFOR).

[26]  Ian McGraw,et al.  Personalized speech recognition on mobile devices , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[27]  Kire Trivodaliev,et al.  A review of Internet of Things for smart home: Challenges and solutions , 2017 .

[28]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[29]  Kire Trivodaliev,et al.  Enabling internet of things for smart homes through fog computing , 2017, 2017 25th Telecommunication Forum (TELFOR).

[30]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.