A Review on the Artificial Intelligence Algorithms for the Recognition of Activities of Daily Living Using Sensors in Mobile Devices

Smart environments and mobile devices are two technologies that when combined may allow the recognition of Activities of Daily Living (ADL) and its environments. This paper focuses on the literature review of the existing machine learning methods for the recognition of ADL and its environments, by means of comparison jointly with a proposal of a novel taxonomy in this context. The sensors used for this purpose depends on the nature of the system and the ADL to recognize. The available in the mobile devices are mainly motion, magnetic and location sensors, but the sensors available in the smart environments may have different types. Data acquired from several sensors can be used for the identification of ADL, where the motion, magnetic and location sensors handle the recognition of activities with movement, and the acoustic sensors handle the recognition of activities related with the environment.

[1]  Anthony J. Maeder,et al.  An Adaptive Rule-Based Approach to Classifying Activities of Daily Living , 2015, 2015 International Conference on Healthcare Informatics.

[2]  Gary M. Weiss,et al.  Activity recognition using cell phone accelerometers , 2011, SKDD.

[3]  J.K. Aggarwal,et al.  Human activity analysis , 2011, ACM Comput. Surv..

[4]  Guang-Zhong Yang,et al.  Real-time food intake classification and energy expenditure estimation on a mobile device , 2015, 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[5]  Khaled Eskaf,et al.  Aggregated Activity Recognition Using Smart Devices , 2016, 2016 3rd International Conference on Soft Computing & Machine Intelligence (ISCMI).

[6]  Yufei Chen,et al.  Performance Analysis of Smartphone-Sensor Behavior for Human Activity Recognition , 2017, IEEE Access.

[7]  Mario Cannataro,et al.  Protein-to-protein interactions: Technologies, databases, and algorithms , 2010, CSUR.

[8]  Mohamed Khalgui,et al.  Introduction to the Special Issue on Modeling and Verification of Discrete Event Systems , 2013, TECS.

[9]  Enamul Hoque,et al.  AALO: Activity recognition in smart homes using Active Learning in the presence of Overlapped activities , 2012, 2012 6th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops.

[10]  Zongjian He,et al.  A wearable wireless body area network for human activity recognition , 2014, 2014 Sixth International Conference on Ubiquitous and Future Networks (ICUFN).

[11]  Nuno M. Garcia,et al.  Limitations of the Use of Mobile Devices and Smart Environments for the Monitoring of Ageing People , 2018, ICT4AWE.

[12]  Ig-Jae Kim,et al.  Activity Recognition Using Wearable Sensors for Elder Care , 2008, 2008 Second International Conference on Future Generation Communication and Networking.

[13]  Nuno M. Garcia A Roadmap to the Design of a Personal Digital Life Coach , 2015, ICT Innovations.

[14]  Shin-Dug Kim,et al.  A Dynamic Approach to Recognize Activities in WSN , 2013, Int. J. Distributed Sens. Networks.

[15]  Nirmalya Roy,et al.  GeSmart: A gestural activity recognition model for predicting behavioral health , 2014, 2014 International Conference on Smart Computing.

[16]  Lin Wu,et al.  A PEFKS- and CP-ABE-Based Distributed Security Scheme in Interest-Centric Opportunistic Networks , 2013, Int. J. Distributed Sens. Networks.

[17]  Girija Chetty,et al.  Body sensor networks for human activity recognition , 2016, 2016 3rd International Conference on Signal Processing and Integrated Networks (SPIN).

[18]  Miguel A. Labrador,et al.  Centinela: A human activity recognition system based on acceleration and vital sign data , 2012, Pervasive Mob. Comput..

[19]  Nuno M. Garcia,et al.  Identification of activities of daily living through data fusion on motion and magnetic sensors embedded on mobile devices , 2018, Pervasive Mob. Comput..

[20]  Bhuvana Ramabhadran,et al.  Multimodal Classification of Activities of Daily Living Inside Smart Homes , 2009, IWANN.

[21]  Weihua Sheng,et al.  Realtime recognition of complex daily activities using dynamic Bayesian network , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Giancarlo Fortino,et al.  Activity-aaService: Cloud-assisted, BSN-based system for physical activity monitoring , 2015, 2015 IEEE 19th International Conference on Computer Supported Cooperative Work in Design (CSCWD).

[23]  Reza Malekian,et al.  Fall detection using machine learning algorithms , 2016, 2016 24th International Conference on Software, Telecommunications and Computer Networks (SoftCOM).

[24]  Sajal K. Das,et al.  HARKE: Human Activity Recognition from Kinetic Energy Harvesting Data in Wearable Devices , 2018, IEEE Transactions on Mobile Computing.

[25]  Joel J. P. C. Rodrigues,et al.  Ambient Assisted Living , 2015 .

[26]  B. Kröse,et al.  Bayesian Activity Recognition in Residence for Elders , 2007 .

[27]  Héctor Pomares,et al.  Daily living activity recognition based on statistical feature quality group selection , 2012, Expert Syst. Appl..

[28]  Fernanda Irrera,et al.  Mobile Devices for the Real-Time Detection of Specific Human Motion Disorders , 2016, IEEE Sensors Journal.

[29]  Bernt Schiele,et al.  ADL recognition based on the combination of RFID and accelerometer sensing , 2008, Pervasive 2008.

[30]  Ilkka Korhonen,et al.  Detection of Daily Activities and Sports With Wearable Sensors in Controlled and Uncontrolled Conditions , 2008, IEEE Transactions on Information Technology in Biomedicine.

[31]  Ciprian Dobre,et al.  Ambient Assisted Living and Enhanced Living Environments: Principles, Technologies and Control , 2016 .

[32]  Weihua Sheng,et al.  Realtime Recognition of Complex Human Daily Activities Using Human Motion and Location Data , 2012, IEEE Transactions on Biomedical Engineering.

[33]  Matthai Philipose,et al.  Unsupervised Activity Recognition Using Automatically Mined Common Sense , 2005, AAAI.

[34]  Nuno M. Garcia,et al.  Approach for the Development of a Framework for the Identification of Activities of Daily Living Using Sensors in Mobile Devices , 2018, Sensors.

[35]  Takuya Maekawa,et al.  Activity recognition with hand-worn magnetic sensors , 2013, Personal and Ubiquitous Computing.

[36]  Edward Sazonov,et al.  Identifying Activity Levels and Steps of People With Stroke Using a Novel Shoe-Based Sensor , 2012, Journal of neurologic physical therapy : JNPT.

[37]  Miwako Doi,et al.  Indoor-outdoor activity recognition by a smartphone , 2012, UbiComp.

[38]  Subhas Mukhopadhyay,et al.  Intelligent Sensing Systems for Measuring Wellness Indices of the Daily Activities for the Elderly , 2012, 2012 Eighth International Conference on Intelligent Environments.

[39]  Filipe Ferreira,et al.  A mobile application to improve the quality of life via exercise , 2016, 2016 IEEE 12th International Conference on Intelligent Computer Communication and Processing (ICCP).

[40]  Sen Zhang,et al.  Detection of Activities by Wireless Sensors for Daily Life Surveillance: Eating and Drinking , 2009, Sensors.

[41]  Takahiro Hara,et al.  A content search system for mobile devices based on user context recognition , 2012, VR.

[42]  Guo-Tan Liao,et al.  HMM machine learning and inference for Activities of Daily Living recognition , 2010, The Journal of Supercomputing.

[43]  Davide Anguita,et al.  A Public Domain Dataset for Human Activity Recognition using Smartphones , 2013, ESANN.

[44]  Alberto G. Bonomi,et al.  Identifying Types of Physical Activity With a Single Accelerometer: Evaluating Laboratory-trained Algorithms in Daily Life , 2011, IEEE Transactions on Biomedical Engineering.

[45]  Viorel Negru,et al.  Activities of daily living and falls recognition and classification from the wearable sensors data , 2017, 2017 E-Health and Bioengineering Conference (EHB).

[46]  Nuno M. Garcia,et al.  Validation Techniques for Sensor Data in Mobile Health Applications , 2016, J. Sensors.

[47]  Chee-Meng Chew,et al.  Motion intent recognition for control of a lower extremity assistive device (LEAD) , 2013, 2013 IEEE International Conference on Mechatronics and Automation.

[48]  Jesse Hoey,et al.  Sensor-Based Activity Recognition , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[49]  Fabio Tozeto Ramos,et al.  Multi-scale Conditional Random Fields for first-person activity recognition , 2014, 2014 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[50]  Prabitha Urwyler,et al.  Recognition of activities of daily living in healthy subjects using two ad-hoc classifiers , 2015, BioMedical Engineering OnLine.

[51]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[52]  David Wetherall,et al.  Recognizing daily activities with RFID-based sensors , 2009, UbiComp.

[53]  Yinghui Zhou,et al.  Two-Phase Activity Recognition with Smartphone Sensors , 2015, 2015 18th International Conference on Network-Based Information Systems.

[54]  Diane J. Cook,et al.  Infrastructure-assisted smartphone-based ADL recognition in multi-inhabitant smart environments , 2013, 2013 IEEE International Conference on Pervasive Computing and Communications (PerCom).

[55]  S Szewcyzk,et al.  Annotating smart environment sensor data for activity learning. , 2009, Technology and health care : official journal of the European Society for Engineering and Medicine.

[56]  Hongnian Yu,et al.  Elderly activities recognition and classification for applications in assisted living , 2013, Expert Syst. Appl..

[57]  Kazuya Takeda,et al.  Development and preliminary analysis of sensor signal database of continuous daily living activity over the long term , 2014, Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific.

[58]  Sung-Bae Cho,et al.  A Dining Context-Aware System with Mobile and Wearable Devices , 2015, 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing.

[59]  Daniel P. Siewiorek,et al.  Activity recognition and monitoring using multiple sensors on different body positions , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[60]  Diane J. Cook,et al.  Simple and Complex Activity Recognition through Smart Phones , 2012, 2012 Eighth International Conference on Intelligent Environments.

[61]  Claudio E. Palazzi,et al.  Movement pattern recognition through smartphone's accelerometer , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[62]  Deva Ramanan,et al.  Detecting activities of daily living in first-person camera views , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[63]  Paulo Menezes,et al.  Features selection for human activity recognition with iPhone inertial sensors , 2013 .

[64]  Araceli Sanchis,et al.  Activity Recognition Using Hybrid Generative/Discriminative Models on Home Environments Using Binary Sensors , 2013, Sensors.

[65]  Jin Wang,et al.  Generative models for automatic recognition of human daily activities from a single triaxial accelerometer , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[66]  Igi Ardiyanto,et al.  Light sport exercise detection based on smartwatch and smartphone using k-Nearest Neighbor and Dynamic Time Warping algorithm , 2016, 2016 8th International Conference on Information Technology and Electrical Engineering (ICITEE).

[67]  Zachary Fitz-Walter,et al.  Simple classification of walking activities using commodity smart phones , 2009, OZCHI '09.

[68]  Nuno M. Garcia,et al.  From Data Acquisition to Data Fusion: A Comprehensive Review and a Roadmap for the Identification of Activities of Daily Living Using Mobile Devices , 2016, Sensors.

[69]  Rossitza Goleva,et al.  Recognition of Activities of Daily Living Based on Environmental Analyses Using Audio Fingerprinting Techniques: A Systematic Review , 2018, Sensors.

[70]  Dario Pompili,et al.  Erratum to: Human motion recognition using a wireless sensor-based wearable system , 2011, Personal and Ubiquitous Computing.

[71]  Nuno M. Garcia,et al.  Identification of Activities of Daily Living Using Sensors Available in off-the-shelf Mobile Devices: Research and Hypothesis , 2016, ISAmI.

[72]  Brian Caulfield,et al.  An investigation into non-invasive physical activity recognition using smartphones , 2012, 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[73]  Mi Zhang,et al.  Human Daily Activity Recognition With Sparse Representation Using Wearable Sensors , 2013, IEEE Journal of Biomedical and Health Informatics.

[74]  U. Naeem,et al.  A Comparison of Two Hidden Markov Approaches to Task Identification in the Home Environment , 2007, 2007 2nd International Conference on Pervasive Computing and Applications.

[75]  Babak A. Farshchian,et al.  A Combined Smartphone and Smartwatch Fall Detection System , 2015, 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing.

[76]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[77]  Einar Snekkenes,et al.  Gait Authentication and Identification Using Wearable Accelerometer Sensor , 2007, 2007 IEEE Workshop on Automatic Identification Advanced Technologies.

[78]  Shengrui Wang,et al.  A Frequent Pattern Mining Approach for ADLs Recognition in Smart Environments , 2011, 2011 IEEE International Conference on Advanced Information Networking and Applications.

[79]  Weihua Sheng,et al.  Recognizing human daily activity using a single inertial sensor , 2010, 2010 8th World Congress on Intelligent Control and Automation.

[80]  Christiane Gresse von Wangenheim,et al.  A Systematic Literature Review on Usability Heuristics for Mobile Phones , 2013, Int. J. Mob. Hum. Comput. Interact..

[81]  Jung-Hsien Chiang,et al.  Pattern analysis in daily physical activity data for personal health management , 2014, Pervasive Mob. Comput..

[82]  Ismail Uysal,et al.  Inertia Based Recognition of Daily Activities with ANNs and Spectrotemporal Features , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[83]  Suhono Harso Supangkat,et al.  Missing data handling using machine learning for human activity recognition on mobile device , 2016, 2016 International Conference on ICT For Smart Society (ICISS).

[84]  Rossitza Goleva,et al.  Improving Activity Recognition Accuracy in Ambient-Assisted Living Systems by Automated Feature Engineering , 2017, IEEE Access.

[85]  Yufei Chen,et al.  On motion-sensor behavior analysis for human-activity recognition via smartphones , 2016, 2016 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA).

[86]  Maria Cristina Canavarro Teixeira,et al.  Android Library for Recognition of Activities of Daily Living: Implementation Considerations, Challenges, and Solutions , 2018 .

[87]  Michel Vacher,et al.  Complete Sound and Speech Recognition System for Health Smart Homes: Application to the Recognition of Activities of Daily Living , 2010 .

[88]  Juan A. Botía Blaya,et al.  Ambient Assisted Living system for in-home monitoring of healthy independent elders , 2012, Expert Syst. Appl..

[89]  Shuangquan Wang,et al.  Wearable accelerometer based extendable activity recognition system , 2010, 2010 IEEE International Conference on Robotics and Automation.

[90]  Hongnian Yu,et al.  Activity classification using a single wrist-worn accelerometer , 2011, 2011 5th International Conference on Software, Knowledge Information, Industrial Management and Applications (SKIMA) Proceedings.

[91]  Muhammad Shoaib Human Activity Recognition Using Heterogeneous Sensors , 2013, UbiComp 2013.

[92]  Miguel A. Labrador,et al.  A mobile platform for real-time human activity recognition , 2012, 2012 IEEE Consumer Communications and Networking Conference (CCNC).

[93]  Franklin W. Olin,et al.  Detecting User Activities using the Accelerometer on Android Smartphones , 2010 .

[94]  Sian Lun Lau,et al.  Movement recognition using the accelerometer in smartphones , 2010, 2010 Future Network & Mobile Summit.

[95]  Christian Peter,et al.  The hearing trousers pocket: activity recognition by alternative sensors , 2011, PETRA '11.

[96]  Keiichi Yasumoto,et al.  A method for recognizing living activities in homes using positioning sensor and power meters , 2015, 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[97]  Tae-Seong Kim,et al.  A Triaxial Accelerometer-Based Physical-Activity Recognition via Augmented-Signal Features and a Hierarchical Recognizer , 2010, IEEE Transactions on Information Technology in Biomedicine.

[98]  M. Amaç Güvensan,et al.  Discriminative time-domain features for activity recognition on a mobile phone , 2014, 2014 IEEE Ninth International Conference on Intelligent Sensors, Sensor Networks and Information Processing (ISSNIP).

[99]  Hakob Sarukhanyan,et al.  ACTIVITY RECOGNITION USING K-NEAREST NEIGHBOR ALGORITHM ON SMARTPHONE WITH TRI-AXIAL ACCELEROMETER , 2012 .

[100]  Hsu-Yang Kung,et al.  Gesture-aware fall detection system: Design and implementation , 2015, 2015 IEEE 5th International Conference on Consumer Electronics - Berlin (ICCE-Berlin).

[101]  Nuno M. Garcia,et al.  Classification techniques on computerized systems to predict and/or to detect Apnea: A systematic review , 2017, Comput. Methods Programs Biomed..

[102]  Ryosuke Shibasaki,et al.  Activity-Aware Map: Identifying Human Daily Activity Pattern Using Mobile Phone Data , 2010, HBU.

[103]  Jin-Hyuk Hong,et al.  An Activity Recognition System for Ambient Assisted Living Environments , 2012, EvAAL.

[104]  Nadeem Javaid,et al.  Evaluation of Human Activity Recognition and Fall Detection Using Android Phone , 2015, 2015 IEEE 29th International Conference on Advanced Information Networking and Applications.

[105]  Juha Röning,et al.  Recognizing Human Activities User-independently on Smartphones Based on Accelerometer Data , 2012, Int. J. Interact. Multim. Artif. Intell..

[106]  Qi Cheng,et al.  Human activity recognition via motion and vision data fusion , 2010, 2010 Conference Record of the Forty Fourth Asilomar Conference on Signals, Systems and Computers.

[107]  Seungmin Rho,et al.  Physical activity recognition using multiple sensors embedded in a wearable device , 2013, TECS.

[108]  Soundararajan Srinivasan,et al.  Multisensor Fusion in Smartphones for Lifestyle Monitoring , 2010, 2010 International Conference on Body Sensor Networks.

[109]  Allen Y. Yang,et al.  Distributed recognition of human actions using wearable motion sensor networks , 2009, J. Ambient Intell. Smart Environ..

[110]  Chen-Khong Tham,et al.  Eating activity primitives detection - a step towards ADL recognition , 2008, HealthCom 2008 - 10th International Conference on e-health Networking, Applications and Services.