Know Yourself: Physical and Psychological Self-Awareness With Lifelog

Self-awareness is an essential concept in physiology and psychology. Accurate overall self-awareness benefits the development and well being of an individual. The previous research studies on self-awareness mainly collect and analyze data in the laboratory environment through questionnaires, user study, or field research study. However, these methods are usually not real-time and unavailable for daily life applications. Therefore, we propose a new direction of utilizing lifelog for self-awareness. Lifelog records about daily activities are used for analysis, prediction, and intervention on individual physical and psychological status, which can be automatically processed in real-time. With the help of lifelog, ordinary people are able to understand their condition more precisely, get effective personal advice about health, and even discover physical and mental abnormalities at an early stage. As the first step on using lifelog for self-awareness, we learn from the traditional machine learning problems, and summarize a schema on data collection, feature extraction, label tagging, and model learning in the lifelog scenario. The schema provides a flexible and privacy-protected method for lifelog applications. Following the schema, four topics were conducted: sleep quality prediction, personality detection, mood detection and prediction, and depression detection. Experiments on real datasets show encouraging results on these topics, revealing the significant relation between daily activity records and physical and psychological self-awareness. In the end, we discuss the experiment results and limitations in detail and propose an application, Lifelog Recorder, for multi-dimensional self-awareness lifelog data collection.

[1]  Alan F. Smeaton,et al.  LifeLogging: Personal Big Data , 2014, Found. Trends Inf. Retr..

[2]  Reza Nejabati,et al.  2016 15th International Conference on Ubiquitous Computing and Communications and 2016 International Symposium on Cyberspace and Security (IUCC-CSS 2016) , 2017, UbiComp 2017.

[3]  Christine Gorman,et al.  Why we sleep. , 2004, Time.

[4]  Makoto P. Kato,et al.  Overview of NTCIR-13 , 2017, NTCIR.

[5]  J. Changeux,et al.  Neuroscience and Biobehavioral Reviews towards a Cognitive Neuroscience of Self-awareness , 2022 .

[6]  Takayuki Itoh,et al.  MALL: a life log based music recommendation system and portable music player , 2014, SAC.

[7]  Nitesh V. Chawla,et al.  Filling Missing Values on Wearable-Sensory Time Series Data , 2020, SDM.

[8]  Cecilia Mascolo,et al.  Sequence Multi-task Learning to Forecast Mental Wellbeing from Sparse Self-reported Data , 2019, KDD.

[9]  Ieee Staff 2013 IEEE International Conference on Image Processing , 2013 .

[10]  A. Tellegen,et al.  PERSONALITY PROCESSES AND INDIVIDUAL DIFFERENCES An Alternative "Description of Personality": The Big-Five Factor Structure , 2022 .

[11]  Dongkyoo Shin,et al.  Development of emotion recognition interface using complex EEG/ECG bio-signal for interactive contents , 2017, Multimedia Tools and Applications.

[12]  Srinivasan Parthasarathy,et al.  Proceedings of the 2010 SIAM International Conference on Data Mining , 2010 .

[13]  D. Berrigan,et al.  Association between Objectively Measured Physical Activity and Mortality in NHANES. , 2016, Medicine & Science in Sports & Exercise.

[14]  Min Zhang,et al.  Big Five Personality Measurement Based on Lifelog , 2017, LTA@MM.

[15]  Masanobu Abe,et al.  Personalized TV-program recommendations based on life log , 2010, 2010 Digest of Technical Papers International Conference on Consumer Electronics (ICCE).

[16]  Takeshi Tanaka,et al.  Your activity tracker knows when you quit smoking , 2014, SEMWEB.

[17]  Wioleta Szwoch,et al.  A Review of Emotion Recognition Methods Based on Data Acquired via Smartphone Sensors , 2020, Sensors.

[18]  Thierry Pun,et al.  DEAP: A Database for Emotion Analysis ;Using Physiological Signals , 2012, IEEE Transactions on Affective Computing.

[19]  Minh-Triet Tran,et al.  Introduction to the Third Annual Lifelog Search Challenge (LSC'20) , 2020, ICMR.

[20]  S. Duval,et al.  A theory of objective self awareness , 1972 .

[21]  C. Moulin,et al.  Benefits of SenseCam review on neuropsychological test performance. , 2013, American journal of preventive medicine.

[22]  Samit Bhattacharya,et al.  Using Deep and Convolutional Neural Networks for Accurate Emotion Classification on DEAP Dataset , 2017, AAAI.

[23]  Michael Wagner,et al.  Eye movement analysis for depression detection , 2013, 2013 IEEE International Conference on Image Processing.

[24]  Cathal Gurrin,et al.  Towards Activity Recommendation from Lifelogs , 2014, iiWAS.

[25]  M B Keller,et al.  Personality and Depression: Empirical Findings , 1983 .

[26]  Daqing Zhang,et al.  Toward a human-friendly assistive environment : ICOST'2004, 2nd International Conference on Smart Homes and Health Telematics , 2004 .

[27]  Jaideep Srivastava,et al.  The Science of Sweet Dreams: Predicting Sleep Efficiency from Wearable Device Data , 2017, Computer.

[28]  Yiqun Liu,et al.  A Multi-level Interactive Lifelog Search Engine with User Feedback , 2020, LSC@ICMR.

[29]  P. Rochat Five levels of self-awareness as they unfold early in life , 2003, Consciousness and Cognition.

[30]  Leo Galway,et al.  Feature Extraction for Emotion Recognition and Modelling Using Neurophysiological Data , 2016, 2016 15th International Conference on Ubiquitous Computing and Communications and 2016 International Symposium on Cyberspace and Security (IUCC-CSS).

[31]  M. Zhang,et al.  Mood Detection and Prediction Based on User Daily Activities , 2018, 2018 First Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia).

[32]  Anastazjia F. Ray,et al.  Physical activity and health , 2019 .

[33]  Jingcheng Du,et al.  Extracting psychiatric stressors for suicide from social media using deep learning , 2018, BMC Medical Informatics and Decision Making.

[34]  Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , 2019, KDD.

[35]  Yuuki Nishiyama,et al.  Toward Health Exercise Behavior Change for Teams Using Lifelog Sharing Models , 2016, IEEE Journal of Biomedical and Health Informatics.

[36]  J. Poon,et al.  MOOD: A REVIEW OF ITS ANTECEDENTS AND CONSEQUENCES , 2001 .

[37]  Sungyoung Lee,et al.  Unhealthy Dietary Behavior Based User Life-Log Monitoring for Wellness Services , 2017, ICOST.

[38]  K. Killian Development and validation of the Emotional Self-Awareness Questionnaire: a measure of emotional intelligence. , 2012, Journal of marital and family therapy.

[39]  Shivakant Mishra,et al.  Having Fun?: Personalized Activity-Based Mood Prediction in Social Media , 2017, Prediction and Inference from Social Networks and Social Media.

[40]  R. Thayer The biopsychology of mood and arousal , 1989 .

[41]  Guillaume Gravier,et al.  Proceedings of the 27th ACM International Conference on Multimedia , 2019, ACM Multimedia.

[42]  Tat-Seng Chua,et al.  Detecting Stress Based on Social Interactions in Social Networks , 2017, IEEE Transactions on Knowledge and Data Engineering.

[43]  Duc-Tien Dang-Nguyen,et al.  LIFER: An Interactive Lifelog Retrieval System , 2018, LSC@ICMR.

[44]  J. Shahani Empirical Findings , 2019, AutoUni – Schriftenreihe.

[45]  Cathal Gurrin,et al.  Virtual Reality Lifelog Explorer: Lifelog Search Challenge at ACM ICMR 2018 , 2018, LSC@ICMR.

[46]  Hao Tang,et al.  Emotion Recognition using Multimodal Residual LSTM Network , 2019, ACM Multimedia.

[47]  David A. Karp,et al.  Living with Depression: Illness and Identity Turning Points , 1994 .

[48]  P. Costa,et al.  A contemplated revision of the NEO Five-Factor Inventory , 2004 .

[49]  Juyong Park,et al.  Mood and Weather: Feeling the Heat? , 2013, ICWSM.

[50]  Andreas Holzinger,et al.  Interactive machine learning for health informatics: when do we need the human-in-the-loop? , 2016, Brain Informatics.

[51]  C. McClung,et al.  How Might Circadian Rhythms Control Mood? Let Me Count the Ways... , 2013, Biological Psychiatry.