A human-centered wearable sensing platform with intelligent automated data annotation capabilities

Wearable computers provide significant opportunities for sensing and data collection in user's natural environment (NE). However, they require both raw data and annotations to train their respective signal processing algorithms. Collecting these annotations is often burdensome for the users. Our proposed methodology leverages the notion of location from nearable sensors in Internet of Things (IoT) platforms and learns users' patterns of behavior without any prior knowledge. It also requests users for annotations and labels only when the algorithms are unable to automatically annotate the data. We validate our proposed approach in the context of diet monitoring, a significant application that often requires considerable user compliance. Our approach improves eating detection accuracy by 2.4% with requested annotations restricted to 20 per day.

[1]  Gregory D. Abowd,et al.  A practical approach for recognizing eating moments with wrist-mounted inertial sensing , 2015, UbiComp.

[2]  H. Skouteris,et al.  Does the burden of the experience sampling method undermine data quality in state body image research? , 2013, Body image.

[3]  Gregory J. Pottie,et al.  Context-driven, Prescription-Based Personal Activity Classification: Methodology, Architecture, and End-to-End Implementation , 2014, IEEE Journal of Biomedical and Health Informatics.

[4]  Zoubin Ghahramani,et al.  Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning , 2015, ICML.

[5]  Eric Horvitz,et al.  Experience sampling for building predictive user models: a comparative study , 2008, CHI.

[6]  Fanglin Chen,et al.  StudentLife: assessing mental health, academic performance and behavioral trends of college students using smartphones , 2014, UbiComp.

[7]  Romit Roy Choudhury,et al.  Micro-Blog: sharing and querying content through mobile phones and social participation , 2008, MobiSys '08.

[8]  Kent Larson,et al.  Activity Recognition in the Home Using Simple and Ubiquitous Sensors , 2004, Pervasive.

[9]  Deborah Estrin,et al.  Improving activity classification for health applications on mobile devices using active and semi-supervised learning , 2010, 2010 4th International Conference on Pervasive Computing Technologies for Healthcare.

[10]  Mark V. Albert,et al.  In-lab versus at-home activity recognition in ambulatory subjects with incomplete spinal cord injury , 2017, Journal of NeuroEngineering and Rehabilitation.

[11]  Donghai Guan,et al.  Activity Recognition Based on Semi-supervised Learning , 2007, 13th IEEE International Conference on Embedded and Real-Time Computing Systems and Applications (RTCSA 2007).

[12]  Ling Bao,et al.  Activity Recognition from User-Annotated Acceleration Data , 2004, Pervasive.

[13]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[14]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[15]  Majid Sarrafzadeh,et al.  Monitoring eating habits using a piezoelectric sensor-based necklace , 2015, Comput. Biol. Medicine.

[16]  Gregory D. Abowd,et al.  EarBit: Using Wearable Sensors to Detect Eating Episodes in Unconstrained Environments , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[17]  Young-Ho Kim,et al.  OmniTrack: A Flexible Self-Tracking Approach Leveraging Semi-Automated Tracking , 2017, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[18]  Roozbeh Jafari,et al.  MotionSynthesis Toolset (MoST): An Open Source Tool and Data Set for Human Motion Data Synthesis and Validation , 2016, IEEE Sensors Journal.

[19]  Fei-Fei Li,et al.  OPTIMOL: Automatic Online Picture Collection via Incremental Model Learning , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Ben Taskar,et al.  Expectation Maximization and Posterior Constraints , 2007, NIPS.

[21]  Andrea Mannini,et al.  Classifier Personalization for Activity Recognition Using Wrist Accelerometers , 2019, IEEE Journal of Biomedical and Health Informatics.

[22]  Gregory D. Abowd,et al.  Students' Experiences with Ecological Momentary Assessment Tools to Report on Emotional Well-being , 2018, Proc. ACM Interact. Mob. Wearable Ubiquitous Technol..

[23]  Yan Zhou,et al.  Democratic co-learning , 2004, 16th IEEE International Conference on Tools with Artificial Intelligence.

[24]  Guoliang Xing,et al.  FamilyLog: A mobile system for monitoring family mealtime activities , 2017, 2017 IEEE International Conference on Pervasive Computing and Communications (PerCom).