Activity Recognition with Evolving Data Streams

Activity recognition aims to provide accurate and opportune information on people’s activities by leveraging sensory data available in today’s sensory rich environments. Nowadays, activity recognition has become an emerging field in the areas of pervasive and ubiquitous computing. A typical activity recognition technique processes data streams that evolve from sensing platforms such as mobile sensors, on body sensors, and/or ambient sensors. This article surveys the two overlapped areas of research of activity recognition and data stream mining. The perspective of this article is to review the adaptation capabilities of activity recognition techniques in streaming environment. Categories of techniques are identified based on different features in both data streams and activity recognition. The pros and cons of the algorithms in each category are analysed, and the possible directions of future research are indicated.

[1]  Bernt Schiele,et al.  Weakly Supervised Recognition of Daily Life Activities with Wearable Sensors , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Kenji Mase,et al.  Activity and Location Recognition Using Wearable Sensors , 2002, IEEE Pervasive Comput..

[3]  Michael L. Littman,et al.  Activity Recognition from Accelerometer Data , 2005, AAAI.

[4]  Mahmoud Reza Hashemi,et al.  A DCT based approach for detecting novelty and concept drift in data streams , 2010, 2010 International Conference of Soft Computing and Pattern Recognition.

[5]  Jennifer Healey,et al.  A Long-Term Evaluation of Sensing Modalities for Activity Recognition , 2007, UbiComp.

[6]  Qiang Yang,et al.  Cross-Domain Co-Extraction of Sentiment and Topic Lexicons , 2012, ACL.

[7]  Daniel P. Siewiorek,et al.  Activity recognition and monitoring using multiple sensors on different body positions , 2006, International Workshop on Wearable and Implantable Body Sensor Networks (BSN'06).

[8]  Philip S. Yu,et al.  Transfer Feature Learning with Joint Distribution Adaptation , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Ricard Gavaldà,et al.  Learning from Time-Changing Data with Adaptive Windowing , 2007, SDM.

[10]  Fei-Fei Li,et al.  OPTIMOL: Automatic Online Picture Collection via Incremental Model Learning , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Craig A. Knoblock,et al.  Selective Sampling with Redundant Views , 2000, AAAI/IAAI.

[12]  Lawrence B. Holder,et al.  Discovering Activities to Recognize and Track in a Smart Environment , 2011, IEEE Transactions on Knowledge and Data Engineering.

[13]  Wei Niu,et al.  Human activity detection and recognition for video surveillance , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[14]  Stephen J. McKenna,et al.  Activity summarisation and fall detection in a supportive home environment , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[15]  Jesús Favela,et al.  Activity Recognition for the Smart Hospital , 2008, IEEE Intelligent Systems.

[16]  Mitja Lustrek,et al.  Fall Detection and Activity Recognition with Machine Learning , 2009, Informatica.

[17]  Latifur Khan,et al.  Facing the reality of data stream classification: coping with scarcity of labeled data , 2012, Knowledge and Information Systems.

[18]  Weng-Keen Wong,et al.  Physical Activity Recognition from Accelerometer Data Using a Multi-Scale Ensemble Method , 2013, IAAI.

[19]  Maayan Harel,et al.  Learning from Multiple Outlooks , 2010, ICML.

[20]  Yiqiang Chen,et al.  Cross-People Mobile-Phone Based Activity Recognition , 2011, IJCAI.

[21]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  Novelty detection algorithm for data streams multi-class problems , 2013, SAC '13.

[22]  R. B. Woodruff,et al.  Know Your Customer: New Approaches to Understanding Customer Value and Satisfaction , 1996 .

[23]  Ming Liu,et al.  Sensor-based human activity recognition system with a multilayered model using time series shapelets , 2015, Knowl. Based Syst..

[24]  Yiqiang Chen,et al.  Cross-mobile ELM based Activity Recognition , 2010 .

[25]  Schahram Dustdar,et al.  Incorporating Unsupervised Learning in Activity Recognition , 2011, Activity Context Representation.

[26]  Geoff Hulten,et al.  Mining high-speed data streams , 2000, KDD '00.

[27]  Paul Lukowicz,et al.  Collecting complex activity datasets in highly rich networked sensor environments , 2010, 2010 Seventh International Conference on Networked Sensing Systems (INSS).

[28]  Donghai Guan,et al.  Activity Recognition Based on Semi-supervised Learning , 2007, 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA 2007).

[29]  Brian Mac Namee,et al.  Drift detection using uncertainty distribution divergence , 2011, 2011 IEEE 11th International Conference on Data Mining Workshops.

[30]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[31]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[32]  Eamonn J. Keogh,et al.  DTW-D: time series semi-supervised learning from a single example , 2013, KDD.

[33]  Ira Assent,et al.  AnyOut: Anytime Outlier Detection on Streaming Data , 2012, DASFAA.

[34]  Marek Penhaker,et al.  Comparison of Classification Algorithms for Physical Activity Recognition , 2013, IBICA.

[35]  Philip S. Yu,et al.  Transfer Learning on Heterogenous Feature Spaces via Spectral Transformation , 2010, 2010 IEEE International Conference on Data Mining.

[36]  Sung-Bae Cho,et al.  Activity recognition with android phone using mixture-of-experts co-trained with labeled and unlabeled data , 2014, Neurocomputing.

[37]  Paul Lukowicz,et al.  Activity Recognition of Assembly Tasks Using Body-Worn Microphones and Accelerometers , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Ling Shao,et al.  A survey on fall detection: Principles and approaches , 2013, Neurocomputing.

[39]  Deokjai Choi,et al.  Personalization in Mobile Activity Recognition System Using K-Medoids Clustering Algorithm , 2013, Int. J. Distributed Sens. Networks.

[40]  Shai Ben-David,et al.  Detecting Change in Data Streams , 2004, VLDB.

[41]  Diane J. Cook,et al.  Activity recognition on streaming sensor data , 2014, Pervasive Mob. Comput..

[42]  J. C. Schlimmer,et al.  Incremental learning from noisy data , 2004, Machine Learning.

[43]  João Gama,et al.  A survey on learning from data streams: current and future trends , 2012, Progress in Artificial Intelligence.

[44]  A. Madansky.  Identification of Outliers , 1988 .

[45]  Arkady B. Zaslavsky,et al.  Context Aware Computing for The Internet of Things: A Survey , 2013, IEEE Communications Surveys & Tutorials.

[46]  Wenyu Zhang,et al.  Multivariate Time Series Classification Using Dynamic Time Warping Template Selection for Human Activity Recognition , 2015, 2015 IEEE Symposium Series on Computational Intelligence.

[47]  Charu C. Aggarwal,et al.  Detecting Recurring and Novel Classes in Concept-Drifting Data Streams , 2011, 2011 IEEE 11th International Conference on Data Mining.

[48]  Dong-Soo Kwon,et al.  Unsupervised clustering for abnormality detection based on the tri-axial accelerometer , 2009, 2009 ICCAS-SICE.

[49]  Deborah Estrin,et al.  PEIR, the personal environmental impact report, as a platform for participatory sensing systems research , 2009, MobiSys '09.

[50]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[51]  Qiang Yang,et al.  A Survey on Transfer Learning , 2010, IEEE Transactions on Knowledge and Data Engineering.

[52]  Henry A. Kautz,et al.  Fine-grained activity recognition by aggregating abstract object usage , 2005, Ninth IEEE International Symposium on Wearable Computers (ISWC'05).

[53]  Ernestina Menasalvas Ruiz,et al.  Mobile Activity Recognition Using Ubiquitous Data Stream Mining , 2012, DaWaK.

[54]  Bhavani M. Thuraisingham,et al.  Classification and Novel Class Detection in Concept-Drifting Data Streams under Time Constraints , 2011, IEEE Transactions on Knowledge and Data Engineering.

[55]  Fei Liu,et al.  HealthyLife: An Activity Recognition System with Smartphone Using Logic-Based Stream Reasoning , 2012, MobiQuitous.

[56]  Gary M. Weiss,et al.  Actitracker: A Smartphone-Based Activity Recognition System for Improving Health and Well-Being , 2016, 2016 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[57]  Ernestina Menasalvas Ruiz,et al.  MARS: A Personalised Mobile Activity Recognition System , 2012, 2012 IEEE 13th International Conference on Mobile Data Management.

[58]  Trevor J. Hastie,et al.  Discriminative vs Informative Learning , 1997, KDD.

[59]  Eamonn J. Keogh,et al.  Time Series Classification under More Realistic Assumptions , 2013, SDM.

[60]  Gary M. Weiss,et al.  Activity recognition using cell phone accelerometers , 2011, SKDD.

[61]  Alexander Gammerman,et al.  Learning by Transduction , 1998, UAI.

[62]  Qiang Yang,et al.  Transferring Multi-device Localization Models using Latent Multi-task Learning , 2008, AAAI.

[63]  V. Vaidehi,et al.  An activity based mobility prediction strategy for next generation wireless networks , 2006, 2006 IFIP International Conference on Wireless and Optical Communications Networks.

[64]  John Herbert,et al.  Smartphone-based Activity Recognition using Hybrid Classifier - Utilizing Cloud Infrastructure for Data Analysis , 2014, PECCS.

[65]  Diane J. Cook,et al.  Mining Sensor Streams for Discovering Human Activity Patterns over Time , 2010, 2010 IEEE International Conference on Data Mining.

[66]  Ming Zeng,et al.  Recognizing new activities with limited training data , 2015, SEMWEB.

[67]  Geoff Holmes,et al.  Active Learning With Drifting Streaming Data , 2014, IEEE Transactions on Neural Networks and Learning Systems.

[68]  Gerhard Widmer,et al.  Learning in the Presence of Concept Drift and Hidden Contexts , 1996, Machine Learning.

[69]  Yiming Yang,et al.  Topic-conditioned novelty detection , 2002, KDD.

[70]  Didier Stricker,et al.  Personalized mobile physical activity recognition , 2013, ISWC '13.

[71]  Qiang Yang,et al.  Adaptive Localization in a Dynamic WiFi Environment through Multi-view Learning , 2007, AAAI.

[72]  Sethuraman Panchanathan,et al.  Multi-source domain adaptation and its application to early detection of fatigue , 2011, KDD.

[73]  Aleksandar Lazarevic,et al.  Incremental Local Outlier Detection for Data Streams , 2007, 2007 IEEE Symposium on Computational Intelligence and Data Mining.

[74]  Jesús S. Aguilar-Ruiz,et al.  Knowledge discovery from data streams , 2009, Intell. Data Anal..

[75]  Eric Horvitz,et al.  Experience sampling for building predictive user models: a comparative study , 2008, CHI.

[76]  Qiang Yang,et al.  Heterogeneous Transfer Learning for Image Classification , 2011, AAAI.

[77]  Jun Yang,et al.  Physical Activity Recognition with Mobile Phones: Challenges, Methods, and Applications , 2010 .

[78]  Bala Srinivasan,et al.  StreamAR: Incremental and Active Learning with Evolving Sensory Data for Activity Recognition , 2012, 2012 IEEE 24th International Conference on Tools with Artificial Intelligence.

[79]  Philip S. Yu,et al.  Detection and Classification of Changes in Evolving Data Streams , 2006, Int. J. Inf. Technol. Decis. Mak..

[80]  Alois Ferscha,et al.  Real-Time Transfer and Evaluation of Activity Recognition Capabilities in an Opportunistic System , 2011 .

[81]  Simon A. Dobson,et al.  USMART , 2014, ACM Trans. Interact. Intell. Syst..

[82]  Gary M. Weiss,et al.  Applications of mobile activity recognition , 2012, UbiComp.

[83]  Charu C. Aggarwal,et al.  Outlier Analysis , 2013, Springer New York.

[84]  David Yarowsky,et al.  Unsupervised Word Sense Disambiguation Rivaling Supervised Methods , 1995, ACL.

[85]  Gert Cauwenberghs,et al.  Incremental and Decremental Support Vector Machine Learning , 2000, NIPS.

[86]  Masashi Sugiyama,et al.  Importance-weighted least-squares probabilistic classifier for covariate shift adaptation with application to human activity recognition , 2012, Neurocomputing.

[87]  Mykola Pechenizkiy,et al.  An Overview of Concept Drift Applications , 2016 .

[88]  Deborah Estrin,et al.  Improving activity classification for health applications on mobile devices using active and semi-supervised learning , 2010, 2010 4th International Conference on Pervasive Computing Technologies for Healthcare.

[89]  Archan Misra,et al.  HARMONI: Context-aware Filtering of Sensor Data for Continuous Remote Health Monitoring , 2008, 2008 Sixth Annual IEEE International Conference on Pervasive Computing and Communications (PerCom).

[90]  Blake Hannaford,et al.  A Hybrid Discriminative/Generative Approach for Modeling Human Activities , 2005, IJCAI.

[91]  William Nick Street,et al.  A streaming ensemble algorithm (SEA) for large-scale classification , 2001, KDD '01.

[92]  Jesús Favela,et al.  Activity-Aware Computing for Healthcare , 2008, IEEE Pervasive Computing.

[93]  Daqing Zhang,et al.  Assisting Elders with Mild Dementia Staying at Home , 2008, 2008 Sixth Annual IEEE International Conference on Pervasive Computing and Communications (PerCom).

[94]  Diane J. Cook,et al.  Transfer learning for activity recognition: a survey , 2013, Knowledge and Information Systems.

[95]  Juha Pärkkä,et al.  Personalization Algorithm for Real-Time Activity Recognition Using PDA, Wireless Motion Bands, and Binary Decision Tree , 2010, IEEE Transactions on Information Technology in Biomedicine.

[96]  Gary M. Weiss,et al.  The Benefits of Personalized Smartphone-Based Activity Recognition Models , 2014, SDM.

[97]  Yi Yao,et al.  Boosting for transfer learning with multiple sources , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[98]  Guang-Zhong Yang,et al.  Semi-supervised segmentation for activity recognition with Multiple Eigenspaces , 2008, 2008 5th International Summer School and Symposium on Medical Devices and Biosensors.

[99]  Xindong Wu,et al.  Mining Recurring Concept Drifts with Limited Labeled Streaming Data , 2010, TIST.

[100]  Anton Dries,et al.  Adaptive concept drift detection , 2009, SDM.

[101]  Ivor W. Tsang,et al.  Domain Adaptation via Transfer Component Analysis , 2009, IEEE Transactions on Neural Networks.

[102]  Dit-Yan Yeung,et al.  Parzen-window network intrusion detectors , 2002, Object recognition supported by user interaction for service robots.

[103]  Eamonn J. Keogh,et al.  Data Editing Techniques to Allow the Application of Distance-Based Outlier Detection to Streams , 2010, 2010 IEEE International Conference on Data Mining.

[104]  Vipin Kumar,et al.  Chapman & Hall/CRC Data Mining and Knowledge Discovery Series , 2008 .

[105]  Thorsten Joachims,et al.  Detecting Concept Drift with Support Vector Machines , 2000, ICML.

[106]  M. Lawton,et al.  Assessment of older people: self-maintaining and instrumental activities of daily living. , 1969, The Gerontologist.

[107]  André Carlos Ponce de Leon Ferreira de Carvalho,et al.  OLINDDA: a cluster-based approach for detecting novelty and concept drift in data streams , 2007, SAC '07.

[108]  Yisheng Dong,et al.  An active learning system for mining time-changing data streams , 2007, Intell. Data Anal..

[109]  Bala Srinivasan,et al.  Adaptive mobile activity recognition system with evolving data streams , 2015, Neurocomputing.

[110]  Charu C. Aggarwal,et al.  Classification and Adaptive Novel Class Detection of Feature-Evolving Data Streams , 2013, IEEE Transactions on Knowledge and Data Engineering.

[111]  Gary M. Weiss,et al.  The Impact of Personalization on Smartphone-Based Activity Recognition , 2012, AAAI 2012.

[112]  Nigel H. Lovell,et al.  Implementation of a real-time human movement classifier using a triaxial accelerometer for ambulatory monitoring , 2006, IEEE Transactions on Information Technology in Biomedicine.

[113]  Geoff Hulten,et al.  Mining time-changing data streams , 2001, KDD '01.

[114]  João Gama,et al.  Learning with Drift Detection , 2004, SBIA.

[115]  Bernt Schiele,et al.  Multi-graph Based Semi-supervised Learning for Activity Recognition , 2009, 2009 International Symposium on Wearable Computers.

[116]  Charu C. Aggarwal,et al.  Stream Classification with Recurring and Novel Class Detection Using Class-Based Ensemble , 2012, 2012 IEEE 12th International Conference on Data Mining.

[117]  Parisa Rashidi,et al.  Using symbolic aggregate approximation (SAX) to visualize activity transitions among older adults. , 2016, Physiological measurement.

[118]  Matthai Philipose,et al.  Unsupervised Activity Recognition Using Automatically Mined Common Sense , 2005, AAAI.

[119]  Ig-Jae Kim,et al.  Mobile health monitoring system based on activity recognition using accelerometer , 2010, Simul. Model. Pract. Theory.

[120]  Ingrid Renz,et al.  Adaptive Information Filtering : Learning Drifting Concepts , 1998 .

[121]  Mirco Musolesi,et al.  Sensing meets mobile social networks: the design, implementation and evaluation of the CenceMe application , 2008, SenSys '08.

[122]  Bernt Schiele,et al.  Exploring semi-supervised and active learning for activity recognition , 2008, 2008 12th IEEE International Symposium on Wearable Computers.

[123]  João Gama,et al.  A survey on concept drift adaptation , 2014, ACM Comput. Surv..

[124]  Bernt Schiele,et al.  Towards Less Supervision in Activity Recognition from Wearable Sensors , 2006, 2006 10th IEEE International Symposium on Wearable Computers.