Emotional Changes Detection for Dementia People with Spectrograms from Physiological Signals

Due to aging society, there has recently been an increasing percentage of people with serious cognitive decline and dementia around the world. Such patients often lose their diversity of facial expressions and even their ability to speak, rendering them unable to express their feelings to their caregivers. However, emotions and feelings are strongly correlated with physiological signals, detectable with EEG and ECG etc. Therefore, this research develops an emotion predicting system for people with dementia using bio-signals to support their interaction with their caregivers. In this paper, we focused on a previous study for binary classification of emotional changes using spectrograms of EEG and RRI by CNN, verifying the effectiveness of the method. Firstly, the participants were required to watch simulating videos while collecting their EEG and ECG data. Then, STFT was performed, processing the raw data signals by extracting the time-frequency domain features to get the spectrograms. Finally, deep learning was used to detect the emotional changes. CNN was used for arousal classification, with an accuracy of 90.00% with EEG spectrograms, 91.67% with RRI spectrograms, and 93.33% with EEG and RRI spectrograms.