Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: State of the art and research challenges

Abstract Human activity recognition systems are developed as part of a framework to enable continuous monitoring of human behaviours in the area of ambient assisted living, sports injury detection, elderly care, rehabilitation, and entertainment and surveillance in smart home environments. The extraction of relevant features is the most challenging part of the mobile and wearable sensor-based human activity recognition pipeline. Feature extraction influences the algorithm performance and reduces computation time and complexity. However, current human activity recognition relies on handcrafted features that are incapable of handling complex activities especially with the current influx of multimodal and high dimensional sensor data. With the emergence of deep learning and increased computation powers, deep learning and artificial intelligence methods are being adopted for automatic feature learning in diverse areas like health, image classification, and recently, for feature extraction and classification of simple and complex human activity recognition in mobile and wearable sensors. Furthermore, the fusion of mobile or wearable sensors and deep learning methods for feature learning provide diversity, offers higher generalisation, and tackles challenging issues in human activity recognition. The focus of this review is to provide in-depth summaries of deep learning methods for mobile and wearable sensor-based human activity recognition. The review presents the methods, uniqueness, advantages and their limitations. We not only categorise the studies into generative, discriminative and hybrid methods but also highlight their important advantages. Furthermore, the review presents classification and evaluation procedures and discusses publicly available datasets for mobile sensor human activity recognition. Finally, we outline and explain some challenges to open research problems that require further research and improvements.

[1]  Plamen Angelov,et al.  A Comprehensive Review on Handcrafted and Learning-Based Action Representation Approaches for Human Activity Recognition , 2017 .

[2]  Christian Igel,et al.  Training restricted Boltzmann machines: An introduction , 2014, Pattern Recognit..

[3]  Héctor Pomares,et al.  mHealthDroid: A Novel Framework for Agile Development of Mobile Health Applications , 2014, IWAAL.

[4]  Wu Liu,et al.  Deep learning hashing for mobile visual search , 2017, EURASIP J. Image Video Process..

[5]  Geoffrey E. Hinton,et al.  Phone recognition using Restricted Boltzmann Machines , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[6]  Tadahiro Taniguchi,et al.  Feature Extraction and Pattern Recognition for Human Motion by a Deep Sparse Autoencoder , 2014, 2014 IEEE International Conference on Computer and Information Technology.

[7]  Sven Tomforde,et al.  Towards Self-Improving Activity Recognition Systems Based on Probabilistic, Generative Models , 2016, 2016 IEEE International Conference on Autonomic Computing (ICAC).

[8]  Faicel Chamroukhi,et al.  Physical Human Activity Recognition Using Wearable Sensors , 2015, Sensors.

[9]  María de Lourdes Martínez-Villaseñor,et al.  A Flexible Approach for Human Activity Recognition Using Artificial Hydrocarbon Networks , 2016, Sensors.

[10]  Min Sheng,et al.  Short-time activity recognition with wearable sensors using convolutional neural network , 2016, VRCAI.

[11]  Lukun Wang,et al.  Recognition of Human Activities Using Continuous Autoencoders with Wearable Sensors , 2016, Sensors.

[12]  Shehroz S. Khan,et al.  Detecting unseen falls from wearable devices using channel-wise ensemble of autoencoders , 2016, Expert Syst. Appl..

[13]  Yoshua Bengio,et al.  Unsupervised and Transfer Learning Challenge: a Deep Learning Approach , 2011, ICML Unsupervised and Transfer Learning.

[14]  Paul J. M. Havinga,et al.  Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors , 2016, Sensors.

[15]  Youngwook Kim,et al.  Human Activity Classification Based on Micro-Doppler Signatures Using a Support Vector Machine , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[16]  Lianghua He,et al.  Deep Learning in the EEG Diagnosis of Alzheimer's Disease , 2014, ACCV Workshops.

[17]  Rossitza Goleva,et al.  Improving Activity Recognition Accuracy in Ambient-Assisted Living Systems by Automated Feature Engineering , 2017, IEEE Access.

[18]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[19]  Junqi Guo,et al.  Motion Recognition by Using a Stacked Autoencoder-Based Deep Learning Algorithm with Smart Phones , 2015, WASA.

[20]  Xiangqian Ding,et al.  A Novel Method for Classification of ECG Arrhythmias Using Deep Belief Networks , 2016, Int. J. Comput. Intell. Appl..

[21]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[22]  Hang Lei,et al.  Sparse codes fusion for context enhancement of night video surveillance , 2015, Multimedia Tools and Applications.

[23]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[24]  Hassan Ghasemzadeh,et al.  Multi-sensor fusion in body sensor networks: State-of-the-art and research challenges , 2017, Inf. Fusion.

[25]  David J. Field,et al.  Sparse coding with an overcomplete basis set: A strategy employed by V1? , 1997, Vision Research.

[26]  Yan Wu,et al.  Automatic sleep stage classification of single-channel EEG by using complex-valued convolutional neural network , 2017, Biomedizinische Technik. Biomedical engineering.

[27]  Marcus Edel,et al.  Binarized-BLSTM-RNN based Human Activity Recognition , 2016, 2016 International Conference on Indoor Positioning and Indoor Navigation (IPIN).

[28]  Y. LeCun,et al.  Learning methods for generic object recognition with invariance to pose and lighting , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[29]  Raphaël Couturier,et al.  Multisensor data fusion and decision support in wireless body sensor networks , 2016, NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium.

[30]  Mark Roantree,et al.  A Framework for Selecting Deep Learning Hyper-parameters , 2015, BICOD.

[31]  Tim Oates,et al.  A Flexible Multichannel EEG Feature Extractor and Classifier for Seizure Detection , 2015, IEEE Transactions on Circuits and Systems II: Express Briefs.

[32]  Geoffrey E. Hinton,et al.  An Efficient Learning Procedure for Deep Boltzmann Machines , 2012, Neural Computation.

[33]  Lei He,et al.  Human activity recognition based on feature selection in smart home using back-propagation algorithm. , 2014, ISA transactions.

[34]  Giancarlo Fortino,et al.  Cloud-based Activity-aaService cyber-physical framework for human activity monitoring in mobility , 2017, Future Gener. Comput. Syst..

[35]  Yun Fu,et al.  Sparse Coding on Local Spatial-Temporal Volumes for Human Action Recognition , 2010, ACCV.

[36]  Sven Behnke,et al.  Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition , 2010, ICANN.

[37]  Bo Ding,et al.  Unsupervised Feature Learning for Human Activity Recognition Using Smartphone Sensors , 2014, MIKE.

[38]  Hwee Pink Tan,et al.  Mobile big data analytics using deep learning and apache spark , 2016, IEEE Network.

[39]  Dimitrios Makris,et al.  One-Shot Learning of Human Activity With an MAP Adapted GMM and Simplex-HMM , 2017, IEEE Transactions on Cybernetics.

[40]  Yann LeCun,et al.  Regularization of Neural Networks using DropConnect , 2013, ICML.

[41]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[42]  Hugo Larochelle,et al.  Efficient Learning of Deep Boltzmann Machines , 2010, AISTATS.

[43]  Yeng Chai Soh,et al.  Using unlabeled acoustic data with locality-constrained linear coding for energy-related activity recognition in buildings , 2015, 2015 IEEE International Conference on Automation Science and Engineering (CASE).

[44]  Gang Wang,et al.  NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Xin-She Yang,et al.  Convolutional Neural Networks Applied for Parkinson's Disease Identification , 2016, Machine Learning for Health Informatics.

[46]  Prasant Mohapatra,et al.  Using Deep Learning for Energy Expenditure Estimation with wearable sensors , 2015, 2015 17th International Conference on E-health Networking, Application & Services (HealthCom).

[47]  Seungjin Choi,et al.  Multi-modal Convolutional Neural Networks for Activity Recognition , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[48]  Mykola Pechenizkiy,et al.  A survey on using domain and contextual knowledge for human activity recognition in video streams , 2016, Expert Syst. Appl..

[49]  Xinyu Wu,et al.  Dimensionality reduction of data sequences for human activity recognition , 2016, Neurocomputing.

[50]  Konstantinos N. Plataniotis,et al.  EEG-based affect states classification using Deep Belief Networks , 2016, 2016 Digital Media Industry & Academic Forum (DMIAF).

[51]  Matjaz Gams,et al.  How Accurately Can Your Wrist Device Recognize Daily Activities and Detect Falls? , 2016, Sensors.

[52]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[53]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[54]  Mahesh K. Marina,et al.  Towards multimodal deep learning for activity recognition on mobile devices , 2016, UbiComp Adjunct.

[55]  Paolo Bonato,et al.  Recent machine learning advancements in sensor-based mobility analysis: Deep learning for Parkinson's disease assessment , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[56]  Bo Yu,et al.  Convolutional Neural Networks for human activity recognition using mobile sensors , 2014, 6th International Conference on Mobile Computing, Applications and Services.

[57]  Naif Alajlan,et al.  Deep learning approach for active classification of electrocardiogram signals , 2016, Inf. Sci..

[58]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[59]  Guang-Zhong Yang,et al.  Deep learning for human activity recognition: A resource efficient implementation on low-power devices , 2016, 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[60]  C. V. Jawahar,et al.  First Person Action Recognition Using Deep Learned Descriptors , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Amy Loutfi,et al.  Sleep Stage Classification Using Unsupervised Feature Learning , 2012, Adv. Artif. Neural Syst..

[62]  Qi Hao,et al.  Convolution neutral network enhanced binary sensor network for human activity recognition , 2016, 2016 IEEE SENSORS.

[63]  Yanjun Qi,et al.  Unsupervised Feature Learning by Deep Sparse Coding , 2013, SDM.

[64]  Mourad Oussalah,et al.  The joint use of sequence features combination and modified weighted SVM for improving daily activity recognition , 2016, Pattern Analysis and Applications.

[65]  Zebin Tahmina,et al.  Human activity recognition with inertial sensors using a deep learning approach , 2016 .

[66]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[67]  Klaus-Robert Müller,et al.  Deep Boltzmann Machines and the Centering Trick , 2012, Neural Networks: Tricks of the Trade.

[68]  Ming-Ai Li,et al.  A novel feature extraction method for scene recognition based on Centered Convolutional Restricted Boltzmann Machines , 2015, Neurocomputing.

[69]  Nuno M. Garcia,et al.  From Data Acquisition to Data Fusion: A Comprehensive Review and a Roadmap for the Identification of Activities of Daily Living Using Mobile Devices , 2016, Sensors.

[70]  Amit K. Roy-Chowdhury,et al.  Continuous Learning of Human Activity Models Using Deep Nets , 2014, ECCV.

[71]  María de Lourdes Martínez-Villaseñor,et al.  A Novel Wearable Sensor-Based Human Activity Recognition Approach Using Artificial Hydrocarbon Networks , 2016, Sensors.

[72]  Nicholas D. Lane,et al.  From smart to deep: Robust activity recognition on smartwatches using deep learning , 2016, 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops).

[73]  Yunpeng Wang,et al.  Long short-term memory neural network for traffic speed prediction using remote microwave sensor data , 2015 .

[74]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[75]  Amit K. Roy-Chowdhury,et al.  A Continuous Learning Framework for Activity Recognition Using Deep Hybrid Feature Models , 2015, IEEE Transactions on Multimedia.

[76]  Benjamin Ghansah,et al.  A Review of Deep Machine Learning , 2016, International Journal of Engineering Research in Africa.

[77]  O. Stegle,et al.  Deep learning for computational biology , 2016, Molecular systems biology.

[78]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[79]  Shiguang Shan,et al.  Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment , 2014, ECCV.

[80]  Xihong Wu,et al.  Recognizing Human Activities from Raw Accelerometer Data Using Deep Neural Networks , 2015, 2015 IEEE 14th International Conference on Machine Learning and Applications (ICMLA).

[81]  Julius Hannink,et al.  Activity recognition in beach volleyball using a Deep Convolutional Neural Network , 2017, Data Mining and Knowledge Discovery.

[82]  Jimeng Sun,et al.  Using recurrent neural network models for early detection of heart failure onset , 2016, J. Am. Medical Informatics Assoc..

[83]  Özlem Durmaz Incel.  Analysis of Movement, Orientation and Rotation-Based Sensing for Phone Placement Recognition , 2015, Sensors.

[84]  Xihong Wu,et al.  Improving activity recognition with context information , 2015, 2015 IEEE International Conference on Mechatronics and Automation (ICMA).

[85]  Gregory D. Abowd,et al.  Predicting daily activities from egocentric images using deep learning , 2015, SEMWEB.

[86]  Athanasios V. Vasilakos,et al.  GCHAR: An efficient Group-based Context - aware human activity recognition on smartphone , 2017, J. Parallel Distributed Comput..

[87]  María de Lourdes Martínez-Villaseñor,et al.  Comparative Analysis of Artificial Hydrocarbon Networks and Data-Driven Approaches for Human Activity Recognition , 2015, UCAmI.

[88]  Patrick Olivier,et al.  Feature Learning for Activity Recognition in Ubiquitous Computing , 2011, IJCAI.

[89]  Yu-Jun Zheng,et al.  Ecogeography-based optimization: Enhancing biogeography-based optimization with ecogeographic barriers and differentiations , 2014, Comput. Oper. Res..

[90]  Branka Jokanovic,et al.  Radar fall motion detection using deep learning , 2016, 2016 IEEE Radar Conference (RadarConf).

[91]  Jake K. Aggarwal,et al.  Human activity recognition from 3D data: A review , 2014, Pattern Recognit. Lett..

[92]  Didier Stricker,et al.  Introducing a New Benchmarked Dataset for Activity Monitoring , 2012, 2012 16th International Symposium on Wearable Computers.

[93]  Tara N. Sainath,et al.  FUNDAMENTAL TECHNOLOGIES IN MODERN SPEECH RECOGNITION Digital Object Identifier 10.1109/MSP.2012.2205597 , 2012 .

[94]  Yang Yi,et al.  Mining human movement evolution for complex action recognition , 2017, Expert Syst. Appl..

[95]  Mi Zhang,et al.  Human Daily Activity Recognition With Sparse Representation Using Wearable Sensors , 2013, IEEE Journal of Biomedical and Health Informatics.

[96]  Limin Sun,et al.  Structural health monitoring by using a sparse coding-based deep learning algorithm with wireless sensor networks , 2014, Personal and Ubiquitous Computing.

[97]  Xihong Wu,et al.  Human activity recognition with HMM-DNN model , 2015, 2015 IEEE 14th International Conference on Cognitive Informatics & Cognitive Computing (ICCI*CC).

[98]  Guilin Chen,et al.  Human Activity Recognition in a Smart Home Environment with Stacked Denoising Autoencoders , 2016, WAIM Workshops.

[99]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[100]  Sven Behnke,et al.  Two-layer contractive encodings for learning stable nonlinear features , 2015, Neural Networks.

[101]  Chen Hu,et al.  Recognizing human activity in smart home using deep learning algorithm , 2014, Proceedings of the 33rd Chinese Control Conference.

[102]  L. Mathew,et al.  Increasing trend of wearables and multimodal interface for human activity monitoring: A review. , 2017, Biosensors & bioelectronics.

[103]  Davide Anguita,et al.  Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine , 2012, IWAAL.

[104]  Aníbal R. Figueiras-Vidal,et al.  On building ensembles of stacked denoising auto-encoding classifiers and their further improvement , 2018, Inf. Fusion.

[105]  Michael S. Lew,et al.  Deep learning for visual understanding: A review , 2016, Neurocomputing.

[106]  Thomas Plötz,et al.  Using unlabeled data in a sparse-coding framework for human activity recognition , 2014, Pervasive Mob. Comput..

[107]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[108]  Gary M. Weiss,et al.  Activity recognition using cell phone accelerometers , 2011, SKDD.

[109]  Jafet Morales,et al.  Physical activity recognition by smartphones, a survey , 2017 .

[110]  Clément Farabet,et al.  Torch7: A Matlab-like Environment for Machine Learning , 2011, NIPS 2011.

[111]  Marc'Aurelio Ranzato,et al.  Efficient Learning of Sparse Representations with an Energy-Based Model , 2006, NIPS.

[112]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[113]  Hong Zhang,et al.  Exploring Techniques for Vision Based Human Activity Recognition: Methods, Systems, and Evaluation , 2013, Sensors.

[114]  Razvan Pascanu,et al.  Theano: A CPU and GPU Math Compiler in Python , 2010, SciPy.

[115]  Brian C. Lovell,et al.  Sparse Coding and Dictionary Learning for Symmetric Positive Definite Matrices: A Kernel Approach , 2012, ECCV.

[116]  Maryam Saeed,et al.  Seizure detection using EEG: A survey of different techniques , 2016, 2016 International Conference on Emerging Technologies (ICET).

[117]  Chalavadi Krishna Mohan,et al.  Hybrid deep neural network model for human action recognition , 2016, Appl. Soft Comput..

[118]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[119]  L. Younes.  On the convergence of markovian stochastic algorithms with rapidly decreasing ergodicity rates , 1999 .

[120]  Peng Wang,et al.  An Adaptive Multi-Sensor Data Fusion Method Based on Deep Convolutional Neural Networks for Fault Diagnosis of Planetary Gearbox , 2017, Sensors.

[121]  Antonio Liotta,et al.  Fusing Bluetooth Beacon Data with Wi-Fi Radiomaps for Improved Indoor Localization , 2017, Sensors.

[122]  Aidong Zhang,et al.  A Novel Semi-Supervised Deep Learning Framework for Affective State Recognition on EEG Signals , 2014, 2014 IEEE International Conference on Bioinformatics and Bioengineering.

[123]  Limin Wang,et al.  Action recognition with trajectory-pooled deep-convolutional descriptors , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[124]  Dong Yu,et al.  Exploring convolutional neural network structures and optimization techniques for speech recognition , 2013, INTERSPEECH.

[125]  Alex Fridman,et al.  Learning Human Identity from Motion Patterns , 2015, IEEE Access.

[126]  Sung-Bae Cho,et al.  Evaluation of Deep Convolutional Neural Network Architectures for Human Activity Recognition with Smartphone Sensors , 2015 .

[127]  Xiaoqiang Lu,et al.  Learning deep event models for crowd anomaly detection , 2017, Neurocomputing.

[128]  Gary D. Bader,et al.  An automated method for finding molecular complexes in large protein interaction networks , 2003, BMC Bioinformatics.

[129]  Mohamad Khalil,et al.  Recognition of different daily living activities using hidden Markov model regression , 2016, 2016 3rd Middle East Conference on Biomedical Engineering (MECBME).

[130]  Robert P. Sheridan,et al.  Deep Neural Nets as a Method for Quantitative Structure-Activity Relationships , 2015, J. Chem. Inf. Model..

[131]  Senem Velipasalar,et al.  A Survey on Activity Detection and Classification Using Wearable Sensors , 2017, IEEE Sensors Journal.

[132]  Gang Ma,et al.  Multi-feature fusion deep networks , 2016, Neurocomputing.

[133]  Guang-Zhong Yang,et al.  A Deep Learning Approach to on-Node Sensor Data Analytics for Mobile or Wearable Devices , 2017, IEEE Journal of Biomedical and Health Informatics.

[134]  Daniel Roggen,et al.  Deep convolutional feature transfer across mobile activity recognition domains, sensor modalities and locations , 2016, SEMWEB.

[135]  Pedro M. Domingos.  A few useful things to know about machine learning , 2012, Commun. ACM.

[136]  Joo-Hwee Lim,et al.  Multimodal Multi-Stream Deep Learning for Egocentric Activity Recognition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[137]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[138]  Yuqing Chen,et al.  A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer , 2015, 2015 IEEE International Conference on Systems, Man, and Cybernetics.

[139]  Adnan Khashman,et al.  Deep learning in vision-based static hand gesture recognition , 2017, Neural Computing and Applications.

[140]  Amy Loutfi,et al.  A review of unsupervised feature learning and deep learning for time-series modeling , 2014, Pattern Recognit. Lett..

[141]  Cesare Furlanello,et al.  Applying Deep Learning to Stereotypical Motor Movement Detection in Autism Spectrum Disorders , 2016, 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW).

[142]  Cem Ersoy,et al.  A Review and Taxonomy of Activity Recognition on Mobile Phones , 2013 .

[143]  Stan Sclaroff,et al.  Learning Activity Progression in LSTMs for Activity Detection and Early Detection , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[144]  Lei Deng,et al.  Temperature based Restricted Boltzmann Machines , 2016, Scientific Reports.

[145]  Jianming Wei,et al.  Toward robust activity recognition: Hierarchical classifier based on Gaussian Process , 2016, Intell. Data Anal..

[146]  Pascal Vincent,et al.  Contractive Auto-Encoders: Explicit Invariance During Feature Extraction , 2011, ICML.

[147]  Mei-rong Zhao,et al.  An evolutionary deep neural network for predicting morbidity of gastrointestinal infections by food contamination , 2017, Neurocomputing.

[148]  David Gijbels,et al.  Deep and surface learning in problem-based learning: a review of the literature , 2015, Advances in Health Sciences Education.

[149]  Jun Zhong,et al.  Towards unsupervised physical activity recognition using smartphone accelerometers , 2016, Multimedia Tools and Applications.

[150]  Li Liu,et al.  Recognizing Complex Activities by a Probabilistic Interval-Based Model , 2016, AAAI.

[151]  Li Liu,et al.  Fusing Social Networks with Deep Learning for Volunteerism Tendency Prediction , 2016, AAAI.

[152]  Bradley J. Erickson,et al.  Toolkits and Libraries for Deep Learning , 2017, Journal of Digital Imaging.

[153]  Sébastien Rebecchi,et al.  An Introduction to Deep Learning , 2011, ESANN.

[154]  Christopher Leckie,et al.  High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning , 2016, Pattern Recognit..

[155]  Weihua Sheng,et al.  Multi-sensor fusion for human daily activity recognition in robot-assisted living , 2009, 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[156]  Jean Ponce,et al.  A Theoretical Analysis of Feature Pooling in Visual Recognition , 2010, ICML.

[157]  Vittorio Rampa,et al.  Device-Free Human Sensing and Localization in Collaborative Human–Robot Workspaces: A Case Study , 2016, IEEE Sensors Journal.

[158]  Tadahiro Kuroda,et al.  Deep learning application trial to lung cancer diagnosis for medical sensor systems , 2016, 2016 International SoC Design Conference (ISOCC).

[159]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[160]  Billur Barshan,et al.  Recognizing Daily and Sports Activities in Two Open Source Machine Learning Environments Using Body-Worn Sensor Units , 2014, Comput. J..

[161]  Meng Wang,et al.  A Deep Structured Model with Radius–Margin Bound for 3D Human Activity Recognition , 2015, International Journal of Computer Vision.

[162]  Diogo R. Ferreira,et al.  Preprocessing techniques for context recognition from accelerometer data , 2010, Personal and Ubiquitous Computing.

[163]  Eiji Oki,et al.  ECG Monitoring System Integrated With IR-UWB Radar Based on CNN , 2016, IEEE Access.

[164]  Geoffrey E. Hinton,et al.  Deep Boltzmann Machines , 2009, AISTATS.

[165]  Sang Min Yoon,et al.  Human activity recognition from accelerometer data using Convolutional Neural Network , 2017, 2017 IEEE International Conference on Big Data and Smart Computing (BigComp).

[166]  Sajal K. Das,et al.  A-Wristocracy: Deep learning on wrist-worn sensing for recognition of user complex activities , 2015, 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[167]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[168]  Ruqiang Yan,et al.  Learning to Monitor Machine Health with Convolutional Bi-Directional LSTM Networks , 2017, Sensors.

[169]  Lior Rokach,et al.  Towards latent context-aware recommendation systems , 2016, Knowl. Based Syst..

[170]  Geoffrey E. Hinton,et al.  Restricted Boltzmann machines for collaborative filtering , 2007, ICML '07.

[171]  Guang-Zhong Yang,et al.  Deep Learning for Health Informatics , 2017, IEEE Journal of Biomedical and Health Informatics.

[172]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[173]  Edward D. Lemaire,et al.  Feature Selection for Wearable Smartphone-Based Human Activity Recognition with Able bodied, Elderly, and Stroke Patients , 2015, PloS one.

[174]  Geoffrey E. Hinton,et al.  Modeling Human Motion Using Binary Latent Variables , 2006, NIPS.

[175]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[176]  Cheng-Yuan Liou,et al.  Autoencoder for words , 2014, Neurocomputing.

[177]  Zhaozheng Yin,et al.  Human Activity Recognition Using Wearable Sensors by Deep Convolutional Neural Networks , 2015, ACM Multimedia.

[178]  Tapani Raiko,et al.  Enhanced Gradient and Adaptive Learning Rate for Training Restricted Boltzmann Machines , 2011, ICML.

[179]  Paul J. M. Havinga,et al.  Fusion of Smartphone Motion Sensors for Physical Activity Recognition , 2014, Sensors.

[180]  Ling Shao,et al.  From handcrafted to learned representations for human action recognition: A survey , 2016, Image Vis. Comput..

[181]  Shafiq R. Joty,et al.  Sleep Quality Prediction From Wearable Data Using Deep Learning , 2016, JMIR mHealth and uHealth.

[182]  Richard Walker,et al.  PD Disease State Assessment in Naturalistic Environments Using Deep Learning , 2015, AAAI.

[183]  Hulya Yalcin.  Human activity recognition using deep belief networks , 2016, 2016 24th Signal Processing and Communication Application Conference (SIU).

[184]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[185]  Jürgen Schmidhuber,et al.  Stacked Convolutional Auto-Encoders for Hierarchical Feature Extraction , 2011, ICANN.

[186]  Christian Vollmer,et al.  Learning Features for Activity Recognition with Shift-Invariant Sparse Coding , 2013, ICANN.

[187]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.

[188]  Xihong Wu,et al.  Real-Time Activity Recognition on Smartphones Using Deep Neural Networks , 2015, 2015 IEEE 12th Intl Conf on Ubiquitous Intelligence and Computing and 2015 IEEE 12th Intl Conf on Autonomic and Trusted Computing and 2015 IEEE 15th Intl Conf on Scalable Computing and Communications and Its Associated Workshops (UIC-ATC-ScalCom).

[189]  Chika Sugimoto,et al.  Recognition of persisting emotional valence from EEG using convolutional neural networks , 2016, 2016 IEEE 9th International Workshop on Computational Intelligence and Applications (IWCIA).

[190]  Antonios Gasteratos,et al.  On-line deep learning method for action recognition , 2014, Pattern Analysis and Applications.

[191]  Abdulsalam Yassine,et al.  Food calorie measurement using deep learning neural network , 2016, 2016 IEEE International Instrumentation and Measurement Technology Conference Proceedings.

[192]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[193]  Claudio Gallicchio,et al.  Human activity recognition using multisensor data fusion based on Reservoir Computing , 2016, J. Ambient Intell. Smart Environ..

[194]  Li Deng,et al.  A tutorial survey of architectures, algorithms, and applications for deep learning , 2014, APSIPA Transactions on Signal and Information Processing.

[195]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[196]  Peter Tiño,et al.  Minimum Complexity Echo State Network , 2011, IEEE Transactions on Neural Networks.

[197]  Lei Wang,et al.  A restricted Boltzmann machine based two-lead electrocardiography classification , 2015, 2015 IEEE 12th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[198]  Zhelong Wang,et al.  Mixed-kernel based weighted extreme learning machine for inertial sensor based human activity recognition with imbalanced dataset , 2016, Neurocomputing.

[199]  Luca Benini,et al.  Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection , 2008, EWSN.

[200]  Rama Chellappa,et al.  Machine Recognition of Human Activities: A Survey , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[201]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[202]  Yoshua Bengio,et al.  Gated Feedback Recurrent Neural Networks , 2015, ICML.

[203]  Alex Zhavoronkov,et al.  Applications of Deep Learning in Biomedicine. , 2016, Molecular pharmaceutics.

[204]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[205]  François Brémond,et al.  Multisensor Fusion for Monitoring Elderly Activities at Home , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[206]  Héctor Pomares,et al.  Multiwindow Fusion for Wearable Activity Recognition , 2015, IWANN.

[207]  Li Cui,et al.  MPD-Model: A Distributed Multipreference-Driven Data Fusion Model and Its Application in a WSNs-Based Healthcare Monitoring System , 2012, Int. J. Distributed Sens. Networks.

[208]  Ahmed Ghoneim,et al.  A Triaxial Accelerometer-Based Human Activity Recognition via EEMD-Based Features and Game-Theory-Based Feature Selection , 2016, IEEE Sensors Journal.

[209]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[210]  Geoffrey E. Hinton,et al.  Learning and relearning in Boltzmann machines , 1986 .

[211]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[212]  Mario Muñoz Organero,et al.  Time-Elastic Generative Model for Acceleration Time Series in Human Activity Recognition , 2017, Sensors.