A smartphone sensors-based personalized human activity recognition system for sustainable smart cities

Abstract According to the Sustainable Development Agenda 2030 of the World Health Organization, maintaining physical activities have multiple societal privileges for healthier cities and societies. The amalgamation of the Internet of Things (IoT) and pervasive smartphones has become of paramount importance to produce a significant breakthrough in various domains of smart cities, including healthcare, fitness, skill assessment, and personal assistants, to support independent living. The IoT-supported devices capacitate, embedded with sensors, enabled numerous context-aware applications to recognize physical activities. There are some activity recognition applications; however, they are still deficient in recognizing activities accurately. In this paper, a novel framework for human activity recognition (HAR) is proposed using raw readings from a combination of fused smartphone sensors: accelerometer, gyroscope, magnetometer, and Google Fit activity tracking module. The proposed framework applies deep recurrent neural network (DRNN) to an extensive training dataset. The latter consists of five activity classes from 12 individuals using a deep recurrent neural network (DRNN). An extensive training dataset is used consisting of five activity classes from a group of 12 individuals. The designed android application (runs in the background) collects data from the smartphone's embedded sensors fused with the Google Fit API to validate the results proposed framework. The proposed framework shows promising results in recognizing human activities compared to other similar studies and achieves an accuracy of 99.43% for activity recognition using DRNN.

[1]  Gary M. Weiss,et al.  Activity recognition using cell phone accelerometers , 2011, SKDD.

[2]  Majid Sarrafzadeh,et al.  Designing a Robust Activity Recognition Framework for Health and Exergaming Using Wearable Sensors , 2014, IEEE Journal of Biomedical and Health Informatics.

[3]  Thar Baker,et al.  Towards fog driven IoT healthcare: challenges and framework of fog computing in healthcare , 2018, ICFNDS.

[4]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[5]  Paul J. M. Havinga,et al.  Fusion of Smartphone Motion Sensors for Physical Activity Recognition , 2014, Sensors.

[6]  Oluwarotimi Williams Samuel,et al.  Adaptive context aware decision computing paradigm for intensive health care delivery in smart cities—A case analysis , 2017, Sustainable Cities and Society.

[7]  Paolo Menaspà.  Effortless activity tracking with Google Fit , 2015, British Journal of Sports Medicine.

[8]  Mohan M. Trivedi,et al.  3-D Posture and Gesture Recognition for Interactivity in Smart Spaces , 2012, IEEE Transactions on Industrial Informatics.

[9]  Muhammad Younus Javed,et al.  Multi-level features fusion and selection for human gait recognition: an optimized framework of Bayesian model and binomial distribution , 2019, Int. J. Mach. Learn. Cybern..

[10]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[11]  Ernesto Damiani,et al.  Privacy-aware Big Data Analytics as a service for public health policies in smart cities , 2018 .

[12]  Angelo M. Sabatini,et al.  Machine Learning Methods for Classifying Human Physical Activity from On-Body Accelerometers , 2010, Sensors.

[13]  Thar Baker,et al.  A collaborative healthcare framework for shared healthcare plan with ambient intelligence , 2020, Hum. centric Comput. Inf. Sci..

[14]  M. Mital,et al.  E-health and wellbeing monitoring using smart healthcare devices: An empirical investigation , 2020, Technological Forecasting and Social Change.

[15]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[16]  Paul J. M. Havinga,et al.  A Survey of Online Activity Recognition Using Mobile Phones , 2015, Sensors.

[17]  Thar Baker,et al.  AlphaLogger: detecting motion-based side-channel attack using smartphone keystrokes , 2020, Journal of Ambient Intelligence and Humanized Computing.

[18]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[19]  Ruixuan Li,et al.  Attentive convolutional gated recurrent network: a contextual model to sentiment analysis , 2020, Int. J. Mach. Learn. Cybern..

[20]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[21]  Ali Kashif Bashir,et al.  PARCIV: Recognizing physical activities having complex interclass variations using semantic data of smartphone , 2020, Softw. Pract. Exp..

[22]  Kim-Kwang Raymond Choo,et al.  Imaging and fusing time series for wearable sensor-based human activity recognition , 2020, Inf. Fusion.

[23]  Muhammad Usman Ghani Khan,et al.  Human activity recognition using mixture of heterogeneous features and sequential minimal optimization , 2018, Int. J. Mach. Learn. Cybern..

[24]  Diane J. Cook,et al.  Learning Setting-Generalized Activity Models for Smart Spaces , 2012, IEEE Intelligent Systems.

[25]  Saif Ur Rehman,et al.  PersonalisedComfort: a personalised thermal comfort model to predict thermal sensation votes for smart building residents , 2020, Enterp. Inf. Syst..

[26]  Abdul Rehman Javed,et al.  Collaborative Health Care Plan through Crowdsource Data using Ambient Application , 2019, 2019 22nd International Multitopic Conference (INMIC).

[27]  Thomas Plötz,et al.  Using unlabeled data in a sparse-coding framework for human activity recognition , 2014, Pervasive Mob. Comput..

[28]  Thar Baker,et al.  Comparison Data Traffic Scheduling Techniques for Classifying QoS over 5G Mobile Networks , 2017, 2017 31st International Conference on Advanced Information Networking and Applications Workshops (WAINA).

[29]  Bhagya Nathali Silva,et al.  Towards sustainable smart cities: A review of trends, architectures, components, and open challenges in smart cities , 2018 .

[30]  Tao Liu,et al.  A hybrid method of recurrent neural network and graph neural network for next-period prescription prediction , 2020, Int. J. Mach. Learn. Cybern..

[31]  Andreas Krause,et al.  Trading off prediction accuracy and power consumption for context-aware wearable computing , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[32]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[33]  Maher Kayal,et al.  Towards the next generation of intelligent building: An assessment study of current automation and future IoT based systems with a proposal for transitional design , 2017 .

[34]  Ejub Kajan,et al.  Immersing citizens and things into smart cities: a social machine-based and data artifact-driven approach , 2020, Computing.

[35]  Jaime Lloret,et al.  An architecture and protocol for smart continuous eHealth monitoring using 5G , 2017, Comput. Networks.

[36]  Jing Zhang,et al.  5G-Smart Diabetes: Toward Personalized Diabetes Diagnosis with Healthcare Big Data Clouds , 2018, IEEE Communications Magazine.

[37]  Sourav Bhattacharyaa,et al.  Towards Using Unlabeled Data in a Sparse-coding Framework for Human Activity Recognition , 2014 .

[38]  Celestine Iwendi,et al.  Analyzing the Effectiveness and Contribution of Each Axis of Tri-Axial Accelerometer Sensor for Accurate Activity Recognition , 2020, Sensors.

[39]  Patrick Olivier,et al.  Feature Learning for Activity Recognition in Ubiquitous Computing , 2011, IJCAI.

[40]  Maureen Schmitter-Edgecombe,et al.  Automated Cognitive Health Assessment Using Smart Home Monitoring of Complex Tasks , 2013, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[41]  Senem Velipasalar,et al.  A Survey on Activity Detection and Classification Using Wearable Sensors , 2017, IEEE Sensors Journal.

[42]  R. Arjmandi,et al.  A composite index for sustainability assessment of health, safety and environmental performance in municipalities of megacities , 2020 .

[43]  Fadi Al-Turjman,et al.  Fog computing for sustainable smart cities in the IoT era: Caching techniques and enabling technologies - an overview , 2020, Sustainable Cities and Society.

[44]  Yonggang Wen,et al.  Multicolumn Bidirectional Long Short-Term Memory for Mobile Devices-Based Human Activity Recognition , 2016, IEEE Internet of Things Journal.

[45]  Hassan Ghasemzadeh,et al.  Physical Movement Monitoring Using Body Sensor Networks: A Phonological Approach to Construct Spatial Decision Trees , 2011, IEEE Transactions on Industrial Informatics.

[46]  Waleed S. Alnumay,et al.  PP-SPA: Privacy Preserved Smartphone-Based Personal Assistant to Improve Routine Life Functioning of Cognitive Impaired Individuals , 2021, Neural Processing Letters.

[47]  Jing Lv,et al.  Recognizing Parkinsonian Gait Pattern by Exploiting Fine-Grained Movement Function Features , 2016, ACM Trans. Intell. Syst. Technol..

[48]  Andrey Ignatov,et al.  Real-time human activity recognition from accelerometer data using Convolutional Neural Networks , 2018, Appl. Soft Comput..

[49]  Miguel A. Labrador,et al.  A Survey on Human Activity Recognition using Wearable Sensors , 2013, IEEE Communications Surveys & Tutorials.

[50]  Paolo Fornacciari,et al.  IoT Wearable Sensor and Deep Learning: An Integrated Approach for Personalized Human Activity Recognition in a Smart Home Environment , 2019, IEEE Internet of Things Journal.

[51]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[52]  Gautam Srivastava,et al.  Automated cognitive health assessment in smart homes using machine learning , 2020 .

[53]  G. Lefebvre,et al.  3D gesture classification with convolutional neural networks , 2014, IEEE International Conference on Acoustics, Speech, and Signal Processing.

[54]  Yu Gu,et al.  PAWS: Passive Human Activity Recognition Based on WiFi Ambient Signals , 2016, IEEE Internet of Things Journal.

[55]  Claudia Eckert,et al.  Neural Network-Based User-Independent Physical Activity Recognition for Mobile Devices , 2015, IDEAL.

[56]  Thar Baker,et al.  IoT-Fog Optimal Workload via Fog Offloading , 2018, 2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion).

[57]  Muhammad Ali Imran,et al.  How 5G Wireless (and Concomitant Technologies) Will Revolutionize Healthcare? , 2017, Future Internet.

[58]  Takeshi Nishida,et al.  Deep recurrent neural network for mobile human activity recognition with high throughput , 2017, Artificial Life and Robotics.