Opportunistic sensing for inferring in-the-wild human contexts based on activity pattern recognition using smart computing

Abstract In recent years, with the evolution of internet-of-things and smart sensing technologies, sensor-based physical activity recognition has gained substantial prominence, and numerous research works have been conducted in this regard. However, the accurate recognition of in-the-wild human activities and the associated contexts remains an open research challenge to be addressed. This research work presents a novel activity-aware human context recognition scheme that explicitly learns human activity patterns in diverse behavioral contexts and infers in-the-wild user contexts based on physical activity recognition. In this aspect, five daily living activities, e.g., lying, sitting, standing, walking, and running, are associated with overall fourteen different behavioral contexts, including phone positions. A public domain dataset, i.e., ExtraSensory, is used for evaluating the proposed scheme using a series of machine learning classifiers. Random Forest classifier achieves the best recognition rate of 88.4% and 89.8% in recognizing five physical activities and the associated behavioral contexts, respectively, which demonstrates the efficacy of the proposed method.

[1]  Leif E. Peterson K-nearest neighbor , 2009, Scholarpedia.

[2]  Ilkka Korhonen,et al.  An Activity Recognition Framework Deploying the Random Forest Classifier and A Single Optical Heart Rate Monitoring and Triaxial Accelerometer Wrist-Band † , 2018, Sensors.

[3]  Jan Nedoma,et al.  Monitoring of the daily living activities in smart home care , 2017, Human-centric Computing and Information Sciences.

[4]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[5]  Jonathan Loo,et al.  Authentication of Smartphone Users Based on Activity Recognition and Mobile Sensing , 2017, Sensors.

[6]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[7]  Harry W. Tyrer,et al.  Context-Aware, Accurate, and Real Time Fall Detection System for Elderly People , 2018, 2018 IEEE 12th International Conference on Semantic Computing (ICSC).

[8]  Hongnian Yu,et al.  A Data Fusion-Based Hybrid Sensory System for Older People’s Daily Activity and Daily Routine Recognition , 2018, IEEE Sensors Journal.

[9]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[10]  Li Zhu,et al.  Smart Systems, the Fourth Industrial Revolution and New Challenges in Distributed Computing , 2017, PARCO.

[11]  Lu Lu,et al.  Activity Recognition in Smart Homes , 2017, Multimedia Tools and Applications.

[12]  Bongwon Suh,et al.  WhichHand: automatic recognition of a smartphone's position in the hand using a smartwatch , 2016, MobileHCI Adjunct.

[13]  Mei-Po Kwan,et al.  Physical activity classification in free-living conditions using smartphone accelerometer data and exploration of predicted results , 2018, Comput. Environ. Urban Syst..

[14]  Farah Ahmad,et al.  Participant experiences in a smartphone-based health coaching intervention for type 2 diabetes: A qualitative inquiry , 2016, Journal of telemedicine and telecare.

[15]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[16]  Muttukrishnan Rajarajan,et al.  Learning models for activity recognition in smart homes , 2015 .

[17]  Max Q.-H. Meng,et al.  A Gait Recognition Method for Human Following in Service Robots , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[18]  Yuan Wu,et al.  A novel orientation- and location-independent activity recognition method , 2017, Personal and Ubiquitous Computing.

[19]  Ramón F. Brena,et al.  Multi-view stacking for activity recognition with sound and accelerometer data , 2018, Inf. Fusion.

[20]  Hadi Tabatabaee Malazi,et al.  PAMS: A new position-aware multi-sensor dataset for human activity recognition using smartphones , 2017, 2017 19th International Symposium on Computer Architecture and Digital Systems (CADS).

[21]  Xiaodong Yang,et al.  Super Normal Vector for Human Activity Recognition with Depth Cameras , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Sankar K. Pal,et al.  Multilayer perceptron, fuzzy sets, and classification , 1992, IEEE Trans. Neural Networks.

[23]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[24]  Minh-Son Dao,et al.  Daily Human Activities Recognition Using Heterogeneous Sensors from Smartphones , 2017 .

[25]  Sajal K. Das,et al.  HuMAn: Complex Activity Recognition with Multi-Modal Multi-Positional Body Sensing , 2019, IEEE Transactions on Mobile Computing.

[26]  Athanasios V. Vasilakos,et al.  GCHAR: An efficient Group-based Context - aware human activity recognition on smartphone , 2017, J. Parallel Distributed Comput..

[27]  Lihui Wang,et al.  Deep learning-based human motion recognition for predictive context-aware human-robot collaboration , 2018 .

[28]  Zhiguang Cao,et al.  Distilling the Knowledge From Handcrafted Features for Human Activity Recognition , 2018, IEEE Transactions on Industrial Informatics.

[29]  Ana M. Bernardos,et al.  Activity logging using lightweight classification techniques in mobile devices , 2012, Personal and Ubiquitous Computing.

[30]  Min-hwa Chi Smart IC technologies for smart devices in IoT applications , 2018, 2018 China Semiconductor Technology International Conference (CSTIC).

[31]  Nadir Weibel,et al.  Context Recognition In-the-Wild , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[32]  Ahmad Almogren,et al.  A robust human activity recognition system using smartphone sensors and deep learning , 2018, Future Gener. Comput. Syst..

[33]  Timo Sztyler,et al.  Position-aware activity recognition with wearable devices , 2017, Pervasive Mob. Comput..

[34]  Shan Chang,et al.  ShakeIn: Secure User Authentication of Smartphones with Single-Handed Shakes , 2017, IEEE Transactions on Mobile Computing.

[35]  Grzegorz J. Nalepa,et al.  Mobile platform for affective context-aware systems , 2019, Future Gener. Comput. Syst..

[36]  Shenghui Zhao,et al.  A Comparative Study on Human Activity Recognition Using Inertial Sensors in a Smartphone , 2016, IEEE Sensors Journal.

[37]  Nasser Kehtarnavaz,et al.  UTD-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and a wearable inertial sensor , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[38]  Iván Pau,et al.  A Context-Aware System Infrastructure for Monitoring Activities of Daily Living in Smart Home , 2016, J. Sensors.

[39]  Teh Ying Wah,et al.  Data fusion and multiple classifier systems for human activity detection and health monitoring: Review and open research directions , 2019, Inf. Fusion.

[40]  Paul J. M. Havinga,et al.  Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors , 2016, Sensors.

[41]  Jonathan Loo,et al.  Continuous authentication of smartphone users based on activity pattern recognition using passive mobile sensing , 2018, J. Netw. Comput. Appl..

[42]  Doruk Coskun,et al.  Phone position/placement detection using accelerometer: Impact on activity recognition , 2015, 2015 IEEE Tenth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP).

[43]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[44]  Chris D. Nugent,et al.  From Activity Recognition to Intention Recognition for Assisted Living Within Smart Homes , 2017, IEEE Transactions on Human-Machine Systems.

[45]  Tm McGinnity,et al.  Activity Recognition from Multi-modal Sensor Data Using a Deep Convolutional Neural Network , 2018 .

[46]  Felipe Barbosa Araújo Ramos,et al.  Combining Smartphone and Smartwatch Sensor Data in Activity Recognition Approaches: an Experimental Evaluation , 2016, SEKE.

[47]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[48]  Jesús Francisco Vargas-Bonilla,et al.  SisFall: A Fall and Movement Dataset , 2017, Sensors.

[49]  Shuang Wang,et al.  A Review on Human Activity Recognition Using Vision-Based Method , 2017, Journal of healthcare engineering.

[50]  Josef Hallberg,et al.  A new approach based on temporal sub-windows for online sensor-based activity recognition , 2018, J. Ambient Intell. Humaniz. Comput..

[51]  Faicel Chamroukhi,et al.  Physical Human Activity Recognition Using Wearable Sensors , 2015, Sensors.

[52]  Konrad P. Kording,et al.  Journal of Neuroscience Methods , 2013 .

[53]  Rahim Tafazolli,et al.  A survey on smartphone-based systems for opportunistic user context recognition , 2013, CSUR.

[54]  Paul J. M. Havinga,et al.  Fusion of Smartphone Motion Sensors for Physical Activity Recognition , 2014, Sensors.

[55]  Shu Wang,et al.  Learning structures of interval-based Bayesian networks in probabilistic generative model for human complex activity recognition , 2018, Pattern Recognit..

[56]  Thar Baker,et al.  Micro-context recognition of sedentary behaviour using smartphone , 2016, 2016 Sixth International Conference on Digital Information and Communication Technology and its Applications (DICTAP).

[57]  Muhammad Awais Azam,et al.  Activity-Aware Fall Detection and Recognition Based on Wearable Sensors , 2019, IEEE Sensors Journal.

[58]  Sergio Escalera,et al.  RGB-D-based Human Motion Recognition with Deep Learning: A Survey , 2017, Comput. Vis. Image Underst..

[59]  Abayomi Moradeyo Otebolaku,et al.  User context recognition using smartphone sensors and classification models , 2016, J. Netw. Comput. Appl..

[60]  Paul Lukowicz,et al.  Opportunistic human activity and context recognition , 2013, Computer.

[61]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[62]  Seok-Won Lee,et al.  Exploratory Data Analysis of Acceleration Signals to Select Light-Weight and Accurate Features for Real-Time Activity Recognition on Smartphones , 2013, Sensors.

[63]  Lu Xu,et al.  Human activity recognition based on random forests , 2017, 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD).

[64]  Xingshe Zhou,et al.  Activity Recognition Using Ubiquitous Sensors: An Overview , 2014 .

[65]  Kenneth N. Brown,et al.  Human activity recognition for emergency first responders via body-worn inertial sensors , 2017, 2017 IEEE 14th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[66]  Mohamed Sedky,et al.  SIMADL: Simulated Activities of Daily Living Dataset , 2018, Data.

[67]  Nirmalya Roy,et al.  Recent trends in machine learning for human activity recognition—A survey , 2018, WIREs Data Mining Knowl. Discov..

[68]  Paul J. M. Havinga,et al.  Towards detection of bad habits by fusing smartphone and smartwatch sensors , 2015, 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[69]  Quang Vinh Nguyen,et al.  RFID Systems in Healthcare Settings and Activity of Daily Living in Smart Homes: A Review , 2017 .

[70]  Francesco Lamonaca,et al.  Health parameters monitoring by smartphone for quality of life improvement , 2015 .

[71]  Young-Koo Lee,et al.  Comprehensive Context Recognizer Based on Multimodal Sensors in a Smartphone , 2012, Sensors.

[72]  Ying Wah Teh,et al.  Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges , 2018, Expert Syst. Appl..

[73]  Uriel Martinez-Hernandez,et al.  Probabilistic identification of sit-to-stand and stand-to-sit with a wearable sensor , 2019, Pattern Recognit. Lett..

[74]  A. Chouhan,et al.  Smart home based ambient assisted living: Recognition of anomaly in the activity of daily living for an elderly living alone , 2018, 2018 IEEE International Instrumentation and Measurement Technology Conference (I2MTC).

[75]  Itzik Klein,et al.  Smartphone Motion Mode Recognition , 2017, ECSA 2017.

[76]  Manolis Tsiknakis,et al.  The MobiAct Dataset: Recognition of Activities of Daily Living using Smartphones , 2016, ICT4AgeingWell.

[77]  Matteo Gadaleta,et al.  IDNet: Smartphone-based Gait Recognition with Convolutional Neural Networks , 2016, Pattern Recognit..