DANA

Current deep neural architectures for processing sensor data are mainly designed for data coming from a fixed set of sensors, with a fixed sampling rate. Changing the dimensions of the input data causes considerable accuracy loss, unnecessary computations, or application failures. To address this problem, we introduce a {\em dimension-adaptive pooling}~(DAP) layer that makes deep architectures robust to temporal changes in sampling rate and in sensor availability. DAP operates on convolutional filter maps of variable dimensions and produces an input of fixed dimensions suitable for feedforward and recurrent layers. Building on this architectural improvement, we propose a {\em dimension-adaptive training}~(DAT) procedure to generalize over the entire space of feasible data dimensions at the inference time. DAT comprises the random selection of dimensions during the forward passes and optimization with accumulated gradients of several backward passes. We then combine DAP and DAT to transform existing non-adaptive deep architectures into a {\em Dimension-Adaptive Neural Architecture}~(DANA) without altering other architectural aspects. Our solution does not need up-sampling or imputation, thus reduces unnecessary computations at inference time. Experimental results on public datasets show that DANA prevents losses in classification accuracy of the state-of-the-art deep architectures, under dynamic sensor availability and varying sampling rates.

[1]  Naser El-Sheimy,et al.  Activity Recognition Using Fusion of Low-Cost Sensors on a Smartphone for Mobile Navigation Application , 2015, Micromachines.

[2]  Mani B. Srivastava,et al.  Privacy risks emerging from the adoption of innocuous wearable sensors in the mobile environment , 2011, CHI.

[3]  J. V. Gemert,et al.  On Translation Invariance in CNNs: Convolutional Layers Can Exploit Absolute Spatial Location , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Hubert Eichner,et al.  Towards Federated Learning at Scale: System Design , 2019, SysML.

[5]  Michael Beigl,et al.  Energy-Efficient Activity Recognition Using Prediction , 2012, 2012 16th International Symposium on Wearable Computers.

[6]  Adrian C. Gleiss,et al.  Assessing the effects of sampling frequency on behavioural classification of accelerometer data , 2019, Journal of Experimental Marine Biology and Ecology.

[7]  Alexander A. Alemi,et al.  On Variational Bounds of Mutual Information , 2019, ICML.

[8]  D. Mohr,et al.  Personal Sensing: Understanding Mental Health Using Ubiquitous Sensors and Machine Learning. , 2017, Annual review of clinical psychology.

[9]  Qing Guo,et al.  Balancing energy, latency and accuracy for mobile sensor data classification , 2011, SenSys.

[10]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[11]  Yingwei Zhang,et al.  Instance-Wise Dynamic Sensor Selection for Human Activity Recognition , 2020, AAAI.

[12]  Joshua Achiam,et al.  On First-Order Meta-Learning Algorithms , 2018, ArXiv.

[13]  G. Clifford,et al.  A review of physiological and behavioral monitoring with digital sensors for neuropsychiatric illnesses , 2018, Physiological measurement.

[14]  R. French Catastrophic forgetting in connectionist networks , 1999, Trends in Cognitive Sciences.

[15]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[16]  Manolis Tsiknakis,et al.  The MobiAct Dataset: Recognition of Activities of Daily Living using Smartphones , 2016, ICT4AgeingWell.

[17]  Rui Wang,et al.  CrossCheck: toward passive sensing and detection of mental health changes in people with schizophrenia , 2016, UbiComp.

[18]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[19]  Yoshua Bengio,et al.  Convolutional networks for images, speech, and time series , 1998 .

[20]  Jeff Gill,et al.  Missing value imputation for physical activity data measured by accelerometer , 2018, Statistical methods in medical research.

[21]  Martin Pielot,et al.  Practical Processing of Mobile Sensor Data for Continual Deep Learning Predictions , 2017, EMDL '17.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  Xingshe Zhou,et al.  Energy-Efficient Motion Related Activity Recognition on Mobile Devices for Pervasive Healthcare , 2014, Mob. Networks Appl..

[24]  J. Schulman,et al.  Reptile: a Scalable Metalearning Algorithm , 2018 .

[25]  Yu Zhao,et al.  Deep Residual Bidir-LSTM for Human Activity Recognition Using Wearable Sensors , 2017, Mathematical Problems in Engineering.

[26]  Davide Anguita,et al.  A Public Domain Dataset for Human Activity Recognition using Smartphones , 2013, ESANN.

[27]  Nicholas D. Lane,et al.  Unsupervised domain adaptation for robust sensory systems , 2019, UbiComp/ISWC Adjunct.

[28]  Vigneshwaran Subbaraju,et al.  Energy-Efficient Continuous Activity Recognition on Mobile Phones: An Activity-Adaptive Approach , 2012, 2012 16th International Symposium on Wearable Computers.

[29]  Matthew Keally,et al.  AdaSense: Adapting sampling rates for activity recognition in Body Sensor Networks , 2013, 2013 IEEE 19th Real-Time and Embedded Technology and Applications Symposium (RTAS).

[30]  Daniel Roggen,et al.  Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition , 2016, Sensors.

[31]  Thomas Plötz,et al.  Optimising sampling rates for accelerometer-based human activity recognition , 2016, Pattern Recognit. Lett..

[32]  Sung-Bae Cho,et al.  Human activity recognition with smartphone sensors using deep learning neural networks , 2016, Expert Syst. Appl..

[33]  Jong-Seok Lee,et al.  EmbraceNet: A robust deep learning architecture for multimodal classification , 2019, Inf. Fusion.

[34]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Andrew E. Myers,et al.  Derivation of body motion via appropriate smoothing of acceleration data , 2008 .

[36]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[37]  Sergey Levine,et al.  Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks , 2017, ICML.

[38]  Daniel Roggen,et al.  Transportation Mode Recognition Fusing Wearable Motion, Sound, and Vision Sensors , 2020, IEEE Sensors Journal.

[39]  Prafulla Kumar Choubey,et al.  Power efficient, bandwidth optimized and fault tolerant sensor management for IOT in Smart Home , 2015, 2015 IEEE International Advance Computing Conference (IACC).

[40]  Gregory W. Corder,et al.  Nonparametric Statistics : A Step-by-Step Approach , 2014 .

[41]  Jiaxing Zhang,et al.  Scale-Invariant Convolutional Neural Networks , 2014, ArXiv.

[42]  Yuan Yu,et al.  TensorFlow: A system for large-scale machine learning , 2016, OSDI.

[43]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Luca Benini,et al.  Activity Recognition from On-Body Sensors: Accuracy-Power Trade-Off by Dynamic Sensor Selection , 2008, EWSN.

[46]  Guihai Chen,et al.  APT: Accurate outdoor pedestrian tracking with smartphones , 2013, 2013 Proceedings IEEE INFOCOM.

[47]  Anthony Winterlich,et al.  Evaluation of sampling frequency, window size and sensor position for classification of sheep behaviour , 2018, Royal Society Open Science.

[48]  Hamed Haddadi,et al.  Privacy and Utility Preserving Sensor-Data Transformations , 2019, Pervasive Mob. Comput..

[49]  Ning Qian,et al.  On the momentum term in gradient descent learning algorithms , 1999, Neural Networks.

[50]  Albrecht Schmidt,et al.  What to Put on the User: Sensing Technologies for Studies and Physiology Aware Systems , 2018, CHI.

[51]  Yoshua Bengio,et al.  Mutual Information Neural Estimation , 2018, ICML.

[52]  M. Srivastava,et al.  SenseHAR: a robust virtual activity sensor for smartphones and wearables , 2019, SenSys.

[53]  Fanglin Chen,et al.  Unobtrusive sleep monitoring using smartphones , 2013, 2013 7th International Conference on Pervasive Computing Technologies for Healthcare and Workshops.

[54]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[55]  Hassan Ghasemzadeh,et al.  Power-Aware Computing in Wearable Sensor Networks: An Optimal Feature Selection , 2015, IEEE Transactions on Mobile Computing.

[56]  Yinghui Zhou,et al.  Two-Phase Activity Recognition with Smartphone Sensors , 2015, 2015 18th International Conference on Network-Based Information Systems.

[57]  Andrea Cavallaro,et al.  Mobile Sensor Data Anonymization , 2019 .

[58]  Andrea Cavallaro,et al.  Protecting Sensory Data against Sensitive Inferences , 2018, P2DS@EuroSys.

[59]  Shaohan Hu,et al.  DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing , 2016, WWW.

[60]  Xiaogang Wang,et al.  Pyramid Scene Parsing Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[62]  Andrey Ignatov,et al.  Real-time human activity recognition from accelerometer data using Convolutional Neural Networks , 2018, Appl. Soft Comput..

[63]  Yunyoung Nam,et al.  Sleep Monitoring Based on a Tri-Axial Accelerometer and a Pressure Sensor , 2016, Sensors.

[64]  Rui Zhang,et al.  Learning Datum-Wise Sampling Frequency for Energy-Efficient Human Activity Recognition , 2018, AAAI.

[65]  Xiaohui Peng,et al.  Deep Learning for Sensor-based Activity Recognition: A Survey , 2017, Pattern Recognit. Lett..

[66]  Kimiaki Shirahama,et al.  A general framework for sensor-based human activity recognition , 2018, Comput. Biol. Medicine.

[67]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[68]  Paul J. M. Havinga,et al.  Complex Human Activity Recognition Using Smartphone and Wrist-Worn Motion Sensors , 2016, Sensors.

[69]  Martin Johns,et al.  Adversarial Preprocessing: Understanding and Preventing Image-Scaling Attacks in Machine Learning , 2020, USENIX Security Symposium.

[70]  Houshang Darabi,et al.  LSTM Fully Convolutional Networks for Time Series Classification , 2017, IEEE Access.

[71]  竹安 数博,et al.  Time series analysis and its applications , 2007 .

[72]  Walid Gomaa,et al.  VersaTL: Versatile Transfer Learning for IMU-based Activity Recognition using Convolutional Neural Networks , 2019, ICINCO.

[73]  Bernt Schiele,et al.  A tutorial on human activity recognition using body-worn inertial sensors , 2014, CSUR.