Automated inference of cognitive performance by fusing multimodal information acquired by smartphone

Recognizing human cognitive performance is important for preserving working efficiency and preventing human error. This paper presents a method for estimating cognitive performance by leveraging multiple information available in a smartphone. The method employs the Go-NoGo task to measure cognitive performance, and fuses contextual and behavioral features to identify the level of performance. It was confirmed that the proposed method could recognize whether cognitive performance was high or low with an average accuracy of 71%, even when only referring to inertial sensor logs. Combining sensing modalities improved the accuracy up to 74%.

[1]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[2]  Dan Cosley,et al.  Mobile manifestations of alertness: connecting biological rhythms with patterns of smartphone app use , 2016, MobileHCI.

[3]  J. Pekar,et al.  Meta-analysis of Go/No-go tasks demonstrating that fMRI activation associated with response inhibition is task-dependent , 2008, Neuropsychologia.

[4]  Olga Sourina,et al.  CogniMeter: EEG-Based Brain States Monitoring , 2016, Trans. Comput. Sci..

[5]  Naoki Yamamoto,et al.  Physiological Stress Level Estimation Based on Smartphone Logs , 2018, 2018 Eleventh International Conference on Mobile Computing and Ubiquitous Network (ICMU).

[6]  Y. Miyashita,et al.  Efficiency of Go/No-Go Task Performance Implemented in the Left Hemisphere , 2012, The Journal of Neuroscience.

[7]  Matthew Kay,et al.  Cognitive rhythms: unobtrusive and continuous sensing of alertness using a mobile phone , 2016, UbiComp.

[8]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[9]  F. Vetere,et al.  Cognitive Heat , 2017 .

[10]  A. Noda,et al.  Poor sleep quality impairs cognitive performance in older adults , 2013, Journal of sleep research.

[11]  D. Dinges,et al.  Psychomotor Vigilance Performance: Neurocognitive Assay Sensitive to Sleep Loss , 2004 .

[12]  D. Dinges,et al.  Psychomotor Vigilance Performance: Neurocognitive Assay Sensitive to Sleep Loss , 2004 .

[13]  Jun Ota,et al.  Predicting anxiety state using smartphone-based passive sensing , 2019, J. Biomed. Informatics.

[14]  Mirco Musolesi,et al.  Trajectories of depression: unobtrusive monitoring of depressive states by means of smartphone mobility traces analysis , 2015, UbiComp.